Skip to content

[Epic] P4 — Multi-Tenant Isolation & Resource Governance (v0.8) #167

Description

@habibtalib

Epic — v0.8 production-hardening, phase P4: Multi-Tenant Isolation & Resource Governance

Goal: one tenant cannot starve, OOM, over-fill, or monopolise a core shared with another; every persisted quota and timeout is actually enforced.

High

  • Per-database/per-tenant max_connections, max_storage_bytes, and memory budgets are dead code — persisted, validated, shown in SHOW QUOTA, but their enforcement functions have no production callers. admission/registry.rs:104,127, security/ceiling.rs:64,79, ddl/database/alter.rs:128-138.
  • Per-tenant in-flight counter leaks on every timeout/panic/over-budget (decrement hook is dead code) → tenant eventually rejected "queue full" with nothing in flight. bridge/dispatch.rs:341-453.
  • statement_timeout accepted but never enforced (only a fixed server-global deadline). pgwire/session/params.rs:119.
  • No mid-execution cancellation — one long query monopolises a single-threaded core and blocks every vShard on it. core_loop/tick.rs:70-84.
  • Graceful shutdown drops in-flight ring requests (DrainingDataPlane unwired). data/runtime.rs:308-402.
  • Hardcoded 1 GiB memory default, no cgroup/RAM detection → governor ceilings mis-sized vs container limit → OS OOM instead of graceful BudgetExhausted. config/server/section.rs:100-102.
  • docker-compose healthcheck broken — curl in a shell-less image against an unregistered /health → always "unhealthy". docker-compose.yml:19-24.

Medium

  • Live quota change resets tracked allocation to 0 → transient up-to-2× over-commit. governor.rs:182-208.
  • WFQ priorities map never reaped → fair-share denominator grows → hair-trigger suspend at depth ≥ 1. nodedb-bridge/src/wfq.rs:244-261.
  • Write-batch coalescing bypasses the 8:4:2 ratio → sustained Critical writes starve reads. tick.rs:130-141.
  • No detection/restart of a dead or DEGRADED core. data/runtime.rs:316-319.
  • No on-disk format migration framework (version mismatch errors, no migrate). No automatic read-only degradation on disk-full.

Note: #153 (spurious auth failures under rapid connects) touches the same connection-governance surface — check for a shared cause.

Tracked in .planning/v1-production-readiness/AUDIT.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions