Skip to content

Per-request tenant isolation, filters and tenant-scoped files - #72

Merged
Iamsdt merged 6 commits into
mainfrom
memory_and_filters
Nov 5, 2026
Merged

Iamsdt merged 6 commits into
mainfrom
memory_and_filters

Conversation

@Iamsdt

@Iamsdt Iamsdt commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Wires the new core multi-tenant storage into the API:

  • The isolation scope is resolved per request from the authenticated user.
  • Thread access compares every column the scope picks (user_id and/or tenant columns such as org_id).
  • Uploaded files are tenant-scoped.
  • Thread and memory routes take safe filters.

Also fixes several isolation edge cases found in review.

Depends on the core PR (PgVectorStore, tenant columns, sql_filter, aget_thread_scope, Isolation). Merge core first.

Why

Core can now isolate threads and memories by developer-declared tenant columns. The API has to:

  • pass each caller's scope to storage
  • make object-level checks (ownership, cache, tombstones, files) honour the same columns
  • expose filtering without opening a path for injection or for widening results

The same user_id in two orgs must behave as two identities on every route.

What changed

Isolation scope per request

  • AuthorizationBackend.isolation_scope(user) is called on every request. Its result is stamped as config["user"]["authz"] through core build_authz.
  • Backends with the old no-argument isolation_scope(self) still work. The method's signature decides how it is called; there is no retry on TypeError.
  • Scope forms:
    • "owner": user_id plus every tenant column.
    • "none": no filter.
    • A column list, for example ["org_id"] for an org admin.
  • Fail closed:
    • An unknown scope string gives a 403.
    • RoleBasedAuthorizationBackend(isolation=...) with a typo refuses to boot.
    • A backend without isolation_scope now means "owner"; before, it meant "none".

Thread ownership

  • OwnershipAuthorizationBackend allows a thread only when the thread's values (aget_thread_scope) equal the caller's on every picked column. The caller's values come only from the authenticated user.
  • The ownership cache stores scope values. Without tenant columns the cache keys are unchanged, so rolling deploys keep working.
  • With tenant columns, cached scopes live in the cache store for 5 minutes instead of 1 hour.
  • The delete tombstone records the deleter's full scope, so the same user_id in another tenant cannot take over a just-deleted thread id.
  • A cache-store failure during eviction is now visible: cleanup_failed: ["cache"].

Filters

  • GET /v1/threads?filters=<JSON>:
    • Limited to 8192 characters.
    • Validated with core validate_filters before any database work.
    • Tenant columns, user_id, unknown operators and bad JSON all answer 422.
  • POST /v1/store/search, /memories/forget and /memories/list validate their filters the same way when the store has a filter schema. Qdrant keeps its old behaviour.
  • 422 messages never echo client values.

Tenant-scoped files

  • With tenant columns declared, each upload records the uploader's tenant values in the file metadata.
  • These must match on:
    • download, /info, /url, delete
    • GET /v1/files
    • quota counting
    • every graph://media/ or file_id reference (invoke, stream, WS, AG-UI, PUT state, POST messages)
    • link_threads
  • An identity without a tenant value cannot upload (403).
  • Files without tenant values are denied while tenant columns are declared. Backfilling them is documented.
  • Deleting a thread, including by an org admin, now releases the thread owner's linked files, not only the deleter's.

Config and loader

  • The loader runs core's tenant-column check on the store bound from the store key in 10xgraph.json. A mismatch with the checkpointer refuses to boot under MODE=production.
    • The opt-out is the new top-level key store_allow_tenant_mismatch, also in the config editor.
  • Custom auth path now accepts a top-level module: auth_app:Cls, auth_app.py:Cls, pkg.mod:Cls and pkg/mod.py:Cls. A missing module still raises.

Other fixes

  • POST /v1/store/search: retrieval_strategy and distance_metric default to None (use the store's settings). They were forced to similarity / cosine, which broke a text-only or HYBRID PgVectorStore, and a Qdrant store using another distance.
  • Mem0 references are removed.
  • Stale client_thread_ids docs are fixed.

Breaking changes

  • An unknown isolation scope fails the request (403), or fails the boot for RoleBasedAuthorizationBackend. It used to become "none" or "owner" silently.
  • A backend without isolation_scope is now isolated per owner, not shared.
  • RoleBasedAuthorizationBackend(isolation="none") now lets any authenticated user reach an existing thread, matching what storage already did under "none".
  • Filters on an extra column of a non-built-in type (for example jsonb) allow only is_null.
  • With tenant columns, files uploaded earlier are denied until their metadata is backfilled.

Testing

  • Full suite: 2861 passed, 9 skipped, coverage 93%.
  • New tests:
    • two orgs with the same user_id isolated on every route family (threads, state, messages, invoke, stream, store, files)
    • forged config.user, config.org_id and config.authz ignored
    • old no-argument backends
    • resolver cache and tombstones across two workers
    • D6 boot refusal and its opt-out
    • each filter rejection
    • the custom auth path forms
  • Mutation checks:
    • breaking the tenant comparison fails 9 isolation tests
    • turning off the file tenant match fails 11 media tests
  • Live check against a running server on real Postgres 17 + pgvector + Redis, with PgCheckpointer and PgVectorStore using org_id and two orgs sharing user sam: 33 of 33 checks pass, covering threads, state, messages, invoke, delete, forged config, filters, memories and files.

Release note

Bump the core dependency, currently 10xgraph>=1.0.0rc4,<2.0, to the core release that ships this feature. The new imports are guarded, so an older core does not crash, but tenant isolation and filters need the new one.

Related

  • core: PgVectorStore, tenant and extra columns, safe filters, Mem0 removal.
  • 10xgraph-client: filters and Filters types.
  • 10xgraph-docs: "Multi-tenant data" and "Use PgVectorStore" guides, plus the configuration reference row for store_allow_tenant_mismatch.

… validation

- Implemented integration tests for tenant isolation in the API, ensuring that users with the same ID in different organizations cannot access each other's threads, states, and messages.
- Added unit tests to validate client filters on extra columns of non-table types, enforcing that only `is_null` is supported for these filters.
- Created tests for the `ThreadOwnershipResolver` to verify caching behavior and correct handling of tenant values.
- Enhanced the authorization backend to ensure proper isolation scope handling and to validate that unauthorized access is correctly denied.
- Updated `OwnershipAuthorizationBackend` to include user context in the `evict` method, allowing for more granular control over ownership checks during thread deletions.
- Introduced `_identity_values` method to standardize the retrieval of user identity values, ensuring consistency in ownership checks.
- Enhanced `ThreadOwnershipResolver` to handle tenant columns more effectively, including a new `evict` method that accommodates both user IDs and tenant values.
- Implemented a mechanism to log and raise errors when cache eviction fails, improving visibility into potential issues with ownership cache management.
- Added a new configuration option `store_allow_tenant_mismatch` to allow for flexibility in tenant column checks during store initialization.
- Developed regression tests to validate the new behavior, ensuring that tenant column checks and cache eviction logic function as intended.
- Introduced tenant value tracking during file uploads, ensuring files are scoped by both user ID and tenant columns.
- Added tenant validation in MediaService methods to enforce access control based on tenant values.
- Updated ensure_can_access and related methods to check tenant values alongside user ownership.
- Enhanced upload_file to raise MissingTenantError when tenant values are not provided.
- Created integration tests to verify tenant isolation behavior, ensuring users from different tenants cannot access each other's files.
- Updated unit tests to accommodate changes in method signatures and ensure correct behavior with tenant values.
@Iamsdt
Iamsdt merged commit a4d225a into main Nov 5, 2026
8 checks passed
@Iamsdt
Iamsdt deleted the memory_and_filters branch November 5, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant