Add cross-repository context sharing to the CLI

[suhaanthayyil]

https://entire.io/gh/entireio/cli/trails/1105

Summary

• add entire context enable, disable, status, query, and inspect
• retrieve authorized evidence per cell with bounded fanout, redaction, sanitization, deterministic merging, and fail-closed completeness checks
• support consent-gated local-live session discovery with serialized cleanup, heartbeat refresh, and bounded evidence retention
• inject untrusted evidence through native Copilot CLI and Factory Droid hook transports while stripping injected blocks from stored transcripts
• add organization context-sharing policy commands and the generated Core API client

Security and reliability

• no raw prompt is sent to Core; scope resolution carries the target repository ID only
• remote queries are redacted and capped; fanout, field sizes, file counts, cache age/count/bytes, and packet delivery are bounded
• partial cell results are rejected instead of silently presented as complete
• evidence is marked untrusted and excluded from transcript persistence
• disable serializes with registry/evidence writers before removing local data

Dependency

• Core policy/API: https://github.com/entirehq/entiredb/pull/3252

Verification

• rebased onto current origin/main
• mise run check (lint plus race-enabled integration suite)
• canary suites: 56/56 and 4/4 passed
• built the CLI and Core feature server, then exercised enable/status/disable over local HTTPS with real CockroachDB, SpiceDB, PostgreSQL, and NATS dependencies
• confirmed final persisted state is false/false/false, disabled scope returns no sources, and audit events contain policy booleans only

[Copilot]

Pull request overview

This PR introduces cross-repository context sharing to the Entire CLI, including new entire context commands and agent-hook based context injection, backed by new Core API endpoints and generated client code.

Changes:

• Add Core API spec + generated client support for context-sharing consent/scope and org policy endpoints.
• Add entire context command group (enable/disable/status/query/inspect) plus org policy commands under entire org context-sharing.
• Implement cross-repo evidence retrieval, sanitization/redaction/bounding, local-live session registry/heartbeat, and injection via Copilot CLI + Factory Droid transports (and strip injected blocks from stored transcripts).

Reviewed changes

Copilot reviewed 40 out of 41 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
internal/coreapi/spec/core.openapi.json	Adds context-sharing schemas and endpoints to the Core OpenAPI spec.
internal/coreapi/spec/core.gen.json	Updates generated spec snapshot for client generation.
internal/coreapi/oas_validators_gen.go	Adds validators for new context-sharing request/response types.
internal/coreapi/oas_security_gen.go	Registers security role maps for new operations.
internal/coreapi/oas_schemas_gen.go	Adds generated Go schema types for context-sharing models.
internal/coreapi/oas_response_decoders_gen.go	Adds generated decoders for new context-sharing endpoints.
internal/coreapi/oas_request_encoders_gen.go	Adds generated encoders for new context-sharing requests.
internal/coreapi/oas_parameters_gen.go	Adds generated params structs for org policy endpoints.
internal/coreapi/oas_operations_gen.go	Adds operation name constants for new endpoints.
internal/coreapi/oas_client_gen.go	Adds generated client methods for consent/scope/org policy endpoints.
cmd/entire/cli/strategy/manual_commit_types.go	Exposes cross-repo context state alias for strategy/session plumbing.
cmd/entire/cli/session/state.go	Persists cross-repo context injection cadence/IDs in session state (no prompt text).
cmd/entire/cli/root.go	Registers new top-level context command group.
cmd/entire/cli/root_test.go	Updates visible-command grouping expectations for context.
cmd/entire/cli/org.go	Adds org context-sharing policy subcommand wiring.
cmd/entire/cli/lifecycle.go	Supports injection-only ContextRequest events and combines legacy + cross-repo injections into one payload.
cmd/entire/cli/corecmd_json_flag_test.go	Extends JSON-flag coverage assertions for org context-sharing subcommands.
cmd/entire/cli/context_retrieval_test.go	Tests bounds/scrubbing/fanout completeness and local-live eligibility rules.
cmd/entire/cli/context_registry.go	Implements local-live session registry storage/locking/retention by (core, account).
cmd/entire/cli/context_registry_test.go	Tests registry namespace isolation, permissions, pruning, and serialized deletion.
cmd/entire/cli/context_injection.go	Implements cross-repo injection cadence, packet rendering, persistence, and local-live registration.
cmd/entire/cli/context_injection_test.go	Tests eligibility, backoff/pending gating, and packet bounds/markers.
cmd/entire/cli/context_cmd.go	Adds entire context enable/disable/status/query/inspect + org policy get/set commands.
cmd/entire/cli/context_cmd_test.go	Verifies command surface and “old Core” error classification behavior.
cmd/entire/cli/attach.go	Ensures stored transcripts are sanitized before downstream processing/redaction.
cmd/entire/cli/agent/transcript_sanitizer_test.go	Adds tests ensuring injected <entire-context> blocks are stripped from stored transcripts.
cmd/entire/cli/agent/inject.go	Extends injection payload to include BaseText for replacement-style transports.
cmd/entire/cli/agent/factoryaidroid/lifecycle.go	Adds context injection support for Factory Droid hook transport.
cmd/entire/cli/agent/factoryaidroid/lifecycle_test.go	Tests Factory Droid injection output shape.
cmd/entire/cli/agent/event.go	Adds ContextRequest event type and TransformedPrompt field for mutation-only hooks.
cmd/entire/cli/agent/copilotcli/types.go	Adds Copilot userPromptTransformed hook config support.
cmd/entire/cli/agent/copilotcli/lifecycle.go	Implements injection-only handling for Copilot transformed prompt (replacement payload).
cmd/entire/cli/agent/copilotcli/lifecycle_test.go	Adds tests for transformed prompt parsing and injection rendering.
cmd/entire/cli/agent/copilotcli/hooks.go	Installs/recognizes the new Copilot transformed hook.
cmd/entire/cli/agent/copilotcli/hooks_test.go	Updates hook install expectations for 9 hooks including transformed prompt.
cmd/entire/cli/agent/copilotcli/compat.go	Parses transformedPrompt field from Copilot hook envelopes.
cmd/entire/cli/agent/capabilities.go	Strips <entire-context> blocks from stored transcripts after agent sanitization.
cmd/entire/cli/agent_help_cmd.go	Classifies new context commands for agent-help surfaces.
cmd/entire/cli/agent_help_banner_test.go	Updates Factory Droid agent-help banner rationale with new injection channel.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[peyton-alt]

Posted here in error. The review findings are tracked only on the Entire trail.

Posted in error; findings are tracked only on the Entire trail.

[suhaanthayyil]

Verification status — replaces the earlier verification prose

The previous "Verification" section overstated what had been run: only the consent layer (enable/status/disable) had ever been executed against a live server. context query, context inspect, cross-jurisdiction fanout, the agent hook injectors, and the local-live registry had never been run as a binary. All of them have now been driven end to end. Two commits of product fixes came out of it.

Harness actually used

• entiredb infra via docker compose --project-name entirehq_entiredb --profile postgres --profile nats --profile core up --detach (CockroachDB, SpiceDB, NATS, Postgres).
• Two entire-core processes over HTTPS from entirehq/entiredb#3252, one carrying a second jurisdiction (local + eu in REGION_HOSTS / JURISDICTION_AUTH_HOSTS / JURISDICTION_API_HOSTS).
• Two entire-api cells, one per jurisdiction.
• Real orgs/projects/repos created through this CLI: ctxtest/alpha (target, local), ctxtest/beta (source, local), ctxtest/gamma (source, eu).

1. entire context query / context inspect — now live

$ entire context query "checkpoint condensation" --json
[ { "id": "ctx_p4ly77v46c7fyuh7", "sourceType": "checkpoint",
    "repoId": "01M0K8XVES26HVZME7MHNMVTBZ", "repoName": "ctxtest/gamma",
    "checkpointId": "cp_benign_0001", "summary": "Condensation rewrites shadow checkpoints…",
    "drillDown": "entire checkpoint explain cp_benign_0001 --repo ctxtest/gamma" } ]

$ entire context inspect ctx_p4ly77v46c7fyuh7
ID / SOURCE / REPOSITORY / CHECKPOINT / SUMMARY / EXCERPT / FILES / DRILL DOWN   (all populated)

$ entire context inspect '../../etc/passwd'
invalid evidence id

Authorized evidence does come back, is persisted, and is re-readable by id.

Fail-closed, each executed for real:

condition	observed
consent revoked (entire context disable)	cross-repository context sharing is disabled; run 'entire context enable', exit 1
caller lacks pull on the target repo	Core 403 permission denied, command fails
target repo unknown	Core 404 target repo not found
after context disable	local namespace removed; context inspect of a previously valid id reports not-found

2. Cross-jurisdiction fanout — now live, with two real cells

Target ctxtest/alpha (local), sources ctxtest/beta (local) and ctxtest/gamma (eu). One context query produced two parallel cell calls:

LOCAL cell :8088  repos=['01M0K7TYTF4GK0ZWR0YZS2SBRP']  q='checkpoint condensation'  auth=bearer-present
EU    cell :8087  repos=['01M0K8XVES26HVZME7MHNMVTBZ']  q='checkpoint condensation'  auth=bearer-present

Each cell received only its own jurisdiction's authorized repo ULID, each got its own per-jurisdiction identity token, the redacted query (not the raw prompt) was what crossed the wire, and evidence from both merged into one result set. Evidence from region B reached a caller in region A.

Org policy gates it as designed: with the org policy at its default deny the eu source was excluded; entire org context-sharing set entire --allow-cross-jurisdiction admitted it; setting it back to deny removed it again.

Bounded-fanout limits remain unit-tested, not live — maxContextScopeSources (640), maxContextFanoutGroups (16), maxContextReposPerRequest (40) are covered by TestContextSourceGroupsChunkAtForty, …RejectsUnboundedScope, …RejectsTooManyConcurrentCells. Only the 2-group case was exercised against real cells; producing 641 sources or 17 cells live was not attempted.

3. Agent injection — now live, and it found four sanitizer bugs

The Copilot CLI and Factory Droid injectors were previously only unit-tested with fakes. Both were driven as real subprocesses with the argv and stdin an actual agent sends:

$ entire hooks copilot-cli user-prompt-transformed   < <real hook JSON>
{"modifiedTransformedPrompt":"BASE-PROMPT>> …\n\n<entire-context>…</entire-context>"}

$ entire hooks factoryai-droid user-prompt-submit    < <real hook JSON>
{"additionalContext":"<entire-context>…</entire-context>"}

Untrusted evidence does land in the prompt, wrapped and labelled untrusted, for both agents, from both local-live and remote-cell sources.

A hostile fixture (literal </entire-context> in mixed case, prompt-injection text, ANSI escapes, NUL and other C0 controls, zero-width and RTL-override characters, oversized fields) was then driven through the same real path. Pre-fix it escaped the packet. Post-fix the emitted packet measures open_delims=1 close_delims=1 packet_ends_closed=True C0/C1=[] Cf=[] has_esc=False has_NUL=False base_prompt_clean=True.

Four product bugs, all in cmd/entire/cli/context_retrieval.go, all reachable from an untrusted transcript or a malicious cell response (fixed in 7ab605a):

1. Panic / hook DoS. stripEntireContextBlocks searched a strings.ToLower copy but sliced the original string with those offsets. Unicode case folding changes byte length (U+023A grows, U+212A and U+0130 shrink), so the offsets addressed the wrong bytes — and on the growth cases panicked with slice bounds out of range inside the evidence goroutine, where no recover can reach it. An untrusted transcript could kill the hook process.
2. Corruption / partial delimiter. The same desync cut the wrong bytes out of evidence. Proven live pre-fix: ȺȺȺ</entire-context>ZORK_AFTER_GROWTH reached the prompt as ȺȺȺ</eK_AFTER_GROWTH.
3. Escape via one NUL byte. Delimiters were stripped before control characters, so a</entire-\x00context>EVIL survived the delimiter pass and then became a real closing tag once the NUL was removed — full packet escape.
4. Escape via unpaired opener. return value[:i] on an unpaired opening tag never rescanned, leaving a closing tag in the output.

Additionally, zero-width and bidi controls (U+200B–U+200F, U+202A–U+202E — the Trojan-Source class) were reaching the model-facing packet, because unicode.IsControl does not cover the Cf category.

The fix replaces the case-folding search with an offset-preserving ASCII fold, strips invisible runes (Cc and Cf) before delimiters, runs the strip loop to a fixed point in one call, and adds a splice backstop so removing one delimiter cannot join surrounding text into a new one — giving a checked postcondition: sanitized text contains neither delimiter in any ASCII casing. It also removes a per-iteration full-string ToLower allocation, making sanitization linear.

Explicitly refuted while looking: the truncation branch in renderCrossRepoContextPacket is unreachable and cannot split a delimiter; Files, DrillDown, and Timestamp never reach the packet; the Copilot base-prompt concatenation cannot be escaped (verified live — the base region was byte-identical).

4. Local-live registry with two concurrent sessions — now live, and it found a fifth bug

Two sessions were registered for two authorized repos, each owned by a real process — one still running, one already exited — both heartbeated inside localContextRecentWindow.

before fix:  local-live ctxtest/beta  session-LIVE
             local-live ctxtest/gamma session-DEAD     <-- a dead agent's transcript, still injected
after  fix:  local-live ctxtest/beta  session-LIVE

localContextSession carried no liveness fingerprint, so "live" meant only "wrote a heartbeat in the last two hours". A crashed or killed agent — or one that exited without a session-end hook — kept being offered as a live context source, and its transcript kept being read and injected into another repo's prompt, for the rest of that window.

Fixed in 4aa063c by recording the same proclive fingerprint (PID + start time + boot + host) the strategy package already uses to reclaim vanished sessions, re-resolved on every heartbeat so a restarted agent is not judged dead by its predecessor's PID. Liveness is deliberately conservative: only LivenessDead excludes an entry; an unknown owner (no fingerprint, another host, a platform that cannot introspect processes) falls back to the recency window, so this cannot silently drop live sessions on Windows or across hosts.

Mutation-verified: disabling the guard makes TestLoadLocalContextEvidenceExcludesDeadSession fail with sessions = [live-session dead-session], want exactly [live-session].

Tests added

TestSanitizeEvidenceTextIsDelimiterFreeAfterOneCall (12 subcases), …SurvivesCaseFoldingLengthChanges, …RemovesBidiAndZeroWidthCharacters, TestStripEntireContextBlocksKeepsSurroundingProse, TestTruncateUTF8BytesRejectsNonPositiveLimits, TestRenderCrossRepoContextPacketSealsHostileEvidence, …StaysSealedWhenTruncated, TestCrossRepoContextInjectionSealsHostileLocalLiveTranscript (registry → transcript → redact → condense → sanitize → packet → both agents' JSON), …RejectsTranscriptOutsideSessionDir, TestLocalContextSessionLiveExcludesDeadOwner, TestLocalContextSessionLiveRejectsExitedProcess (spawns and reaps a real process), TestRefreshContextRegistrySessionRefreshesOwner, TestLoadLocalContextEvidenceExcludesDeadSession.

Still unit-tested only

• The fanout bounds (640 sources / 16 groups / 40 repos per request) — see above.
• Cadence and backoff (crossRepoContextEligible: 2-minute success cadence, Jaccard topic-change threshold, pending lease, 10-minute failure backoff, 4-packet cap).
• Evidence pruning by age/count/bytes (pruneContextEvidenceFiles).
• Multi-cell merge ordering (mergeSemanticV4Responses) beyond the 2-cell case.

Harness limits, stated plainly

• The cells are not real entire-api deployments. entire-api only proxies /semantic-search to a separate entire-search (query-serve) service via SEMANTIC_SEARCH_INTERNAL_URL, and that service is not runnable locally. Each cell here was a server speaking the documented v4 contract (GET /api/v1/semantic-search/search/v1/search?q=&repo=<ULID>) and returning the real wire format. So the CLI half — scope, grouping, per-jurisdiction token minting, parallel fanout, merge, sanitization, persistence, inspect — is genuinely exercised; the cell's own retrieval quality is not.
• No data plane. entire-server/repoadmin was not run, so repo provisioning cannot complete locally and repo_registry placement rows were seeded directly to stand in for a completed provision. Environmental; no code path under test is affected.
• The second jurisdiction needed a rebuilt Core. entiredb's cluster catalog is //go:embed-ed and accepts only local/staging/prod, so adding an eu cluster meant editing the local fixture and rebuilding entire-core. That edit was reverted and is not part of either branch.

Known behaviour left unchanged, flagged for a reviewer

retrieveContextEvidence aborts the entire retrieval on any cell-fanout error, discarding purely-local live evidence that never needed a cell, and in the hook path arms a 10-minute failure backoff. That conflates availability with authorization: one unreachable region silently disables local-live context. It is consistent with the "reject partial results rather than present them as complete" rule this PR is built on, so it was not changed here — relaxing it is a product decision about whether local-live evidence is in scope of that rule, and should be made deliberately rather than as a side effect of this pass.

Gates

mise run check — mise run fmt clean, mise run lint 0 issues, mise run test:ci green: 9429 unit/integration tests, 4 skipped, E2E canary 56/56 (vogon) and 4/4 (roger-roger). Real-agent E2E was not run (costs money).