Skip to content

feat(connectors): establish Phase 5.1 connector interface and capability model - #43

Merged
krishanth7 merged 1 commit into
mainfrom
feat/phase-5.1-connector-interface
Oct 27, 2026
Merged

krishanth7 merged 1 commit into
mainfrom
feat/phase-5.1-connector-interface

Conversation

@krishanth7

Copy link
Copy Markdown
Owner

Executive Summary

Phase 5.1 delivers DERA's universal database connector foundation (packages/connectors/src/dera_connectors/). It defines the typed asynchronous DatabaseConnector protocol, a granular capability discovery taxonomy, universal metadata models with deterministic sorting, connection lifecycle management, sanitized error classification, read-only security enforcement, SSRF egress defenses, an explicit driver registry, a deterministic mock connector (FakeDatabaseConnector), and a reusable conformance test suite (BaseConnectorConformanceTests).

Strict invariants are maintained:

  • Zero AI/LLM models or stochastic inference.
  • Read-only metadata inspection: no customer database modification, no arbitrary SQL execution interface.
  • Tenant isolation: workspace-scoped context and access control.
  • Credential masking: zero secrets in logs, exceptions, or serializations.

Repository Baseline

  • Base Commit SHA: 82071dbe4bb070f98ac5fc690223e8f003d9c7c4
  • Baseline Audit: Recorded in docs/audit/PHASE_5_1_CONNECTOR_BASELINE.md.

Connector Architecture & Layers

  1. Layer A — Platform Infrastructure: FastAPI, Auth/RBAC, PostgreSQL persistence, Celery background jobs.
  2. Layer B — Universal Connector Abstraction (Phase 5.1): Protocol, capabilities, lifecycle, metadata contracts, security, error taxonomy, and testing harness.
  3. Layer C — Database-Specific Adapters (Upcoming Milestones):
    • Phase 5.2: PostgreSQL & Supabase PostgreSQL
    • Phase 5.3: SQLite
    • Phase 5.4: MySQL & MariaDB

Key Deliverables & Modules

  • Protocol (dera_connectors/protocol.py): Strongly typed DatabaseConnector protocol with async context management.
  • Engine Identity (dera_connectors/engine.py): DatabaseEngine (postgresql, sqlite, mysql, mariadb), DatabaseProvider (supabase, aws_rds, gcp_cloudsql, etc.), and sanitized DatabaseIdentity.
  • Capability Model (dera_connectors/capabilities.py): 17 discrete capabilities (SCHEMA_ENUMERATION, TABLE_ENUMERATION, COLUMN_INSPECTION, etc.) and 4 status values (SUPPORTED, UNSUPPORTED, RESTRICTED, UNKNOWN).
  • Metadata Contracts (dera_connectors/metadata.py): Pydantic v2 models for schemas, tables, columns, primary keys, foreign keys, unique constraints, check constraints, indexes, and views with deterministic ordering.
  • Configuration & Credential Boundaries (dera_connectors/configuration.py): ConnectorConfig, TLSSettings, TimeoutSettings, SecretStr password protection, and safe_dict() secret masking.
  • Lifecycle Management (dera_connectors/lifecycle.py): ConnectionLifecycle enforcing explicit transitions (CREATED -> CONNECTING -> CONNECTED -> CLOSING -> CLOSED).
  • Security & SSRF Safeguards (dera_connectors/security.py): validate_read_only_sql blocking mutation keywords; validate_network_target blocking loopback, link-local, and cloud metadata endpoints (169.254.169.254, metadata.google.internal).
  • Error Taxonomy (dera_connectors/errors.py): Standardized error hierarchy (CONNECTOR_UNAVAILABLE, CONNECTOR_AUTHENTICATION_FAILED, CONNECTOR_PERMISSION_DENIED, etc.) with automatic secret masking.
  • Registry & Factory (dera_connectors/registry.py): Explicit ConnectorRegistry and create_connector() factory avoiding dynamic arbitrary imports.
  • Mock Connector (dera_connectors/testing/fake_connector.py): Deterministic mock implementing the protocol for unit testing.
  • Conformance Suite (dera_connectors/testing/conformance.py): BaseConnectorConformanceTests validating protocol conformance, lifecycle, ordering, and security across future adapters.
  • API Contracts (dera_api/schemas/connectors.py): Pydantic schemas for engine discovery, capability matrices, connection test results, and metadata inspection.

Test Results & Telemetry

  • Total Tests Executed: 341
  • Passed: 341
  • Failures: 0
  • Duration: 55.608s
  • Linting & Formatting: 100% compliant with Ruff (python -m ruff check . passed)
  • OpenAPI Schema Drift: 0 drift detected

Documentation Changes

  • docs/connectors/README.md
  • docs/connectors/ARCHITECTURE.md
  • docs/connectors/CONNECTOR_PROTOCOL.md
  • docs/connectors/CAPABILITY_MODEL.md
  • docs/connectors/METADATA_CONTRACTS.md
  • docs/connectors/CONNECTION_LIFECYCLE.md
  • docs/connectors/READ_ONLY_SECURITY.md
  • docs/connectors/CREDENTIAL_HANDLING.md
  • docs/connectors/ERROR_MODEL.md
  • docs/connectors/CONFORMANCE_TESTING.md
  • docs/connectors/ADAPTER_DEVELOPMENT.md
  • docs/connectors/ENGINE_COMPATIBILITY.md
  • docs/audit/PHASE_5_1_CONNECTOR_BASELINE.md
  • docs/audit/PHASE_5_1_CONNECTOR_ACCEPTANCE.md
  • docs/audit/README.md

Acceptance Audit & Phase 5.2 Handoff

  • Audit Verdict: ACCEPTED (PASS) in docs/audit/PHASE_5_1_CONNECTOR_ACCEPTANCE.md.
  • Known Limitations: Production database drivers are intentionally omitted from this milestone and belong to Phases 5.2–5.4.
  • Phase 5.2 Readiness: The universal protocol, capability model, and conformance harness are fully prepared for the PostgreSQL & Supabase adapter implementation.

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@krishanth7
krishanth7 merged commit f94797d into main Oct 27, 2026
11 of 16 checks passed
@krishanth7
krishanth7 deleted the feat/phase-5.1-connector-interface branch October 27, 2026 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant