diff --git a/codegraph-skill/codegraph/SKILL.md b/codegraph-skill/codegraph/SKILL.md index f614ddb9..1fb8ebc4 100644 --- a/codegraph-skill/codegraph/SKILL.md +++ b/codegraph-skill/codegraph/SKILL.md @@ -65,7 +65,9 @@ Treat duplicate leads and call-compatibility hints as review leads, not proof. ## MCP If MCP tools are available, prefer them over repeated CLI invocations. -Use MCP `orient`, `search`, `packet_get`, `goto`, `refs`, `deps`, `rdeps`, `path`, `impact`, and `review` first. +Use MCP `orient`, `search`, `packet_get`, `goto`, `refs`, `deps`, `rdeps`, `path`, `impact`, `review`, and `query_sqlite` first. +After edits, check MCP response `freshness`: `refreshed` means Codegraph rebuilt before answering, and `stale` includes a reason plus bounded changed-file metadata before indexed context is trusted. +Run `refresh_index` before `artifact_build` when MCP reports a stale index; artifact writes refuse stale snapshots. Fall back to CLI when MCP is unavailable. ## Discovery diff --git a/docs/cli.md b/docs/cli.md index 413d3304..d266021b 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -263,11 +263,12 @@ For SQL, prefer handles or schema-qualified names when basenames may be ambiguou - `mcp serve` exposes navigation, search, impact, review, SQLite query, session refresh, and artifact-build tools. - MCP uses stdio by default or Streamable HTTP with `--port `. - Startup is lazy by default; `--warmup` builds the base session cache before serving requests, and `--warmup-symbols` also builds the detailed symbol graph. -- Use `refresh_index` after changing files while a long-running server is active. +- Index-backed responses include `freshness`; small file changes auto-refresh, while stale responses include a reason, total changed-file count, and a bounded changed-file sample. +- Use `refresh_index` to force a rebuild, reset SQLite artifact state, or recover after stale change bursts. - HTTP serves `/mcp`, validates Host headers, and binds to `127.0.0.1` unless `--host ` is passed. - MCP file and artifact paths are confined to `--root` after realpath resolution. -- MCP tools are read-only by default; `--allow-build` enables artifact output only. -- `query_sqlite` is row- and byte-bounded and rejects synthetic payload functions. +- MCP tools are read-only by default; `--allow-build` enables artifact output only when the MCP index is fresh or auto-refreshed. +- `query_sqlite` is row- and byte-bounded, returns freshness metadata, rejects synthetic payload functions, and refuses stale artifact rows it cannot refresh safely. See [docs/mcp.md](./mcp.md) for client configuration examples. diff --git a/docs/library-api.md b/docs/library-api.md index e16e58a8..48c6fb26 100644 --- a/docs/library-api.md +++ b/docs/library-api.md @@ -162,7 +162,9 @@ console.log(artifact.manifestPath, artifact.artifacts); The `graph.json` artifact is self-describing (`schemaVersion: 1`, `format: "codegraph.graph-json"`) and uses project-relative file paths and portable symbol handles. `questions.json` uses the same stable handles for follow-up commands. With `force: true`, stale known Codegraph artifact files are removed before the selected outputs are written; unrelated files in the directory are preserved. -`createAgentSession()` keeps one in-process project snapshot warm for repeated orient, search, explain, packet, artifact, and MCP calls. It uses incremental indexing with disk cache by default, auto-enables native workers for large cold builds, and carries forward top-level analysis metadata from the build report. Set `buildOptions.useNativeWorkers` to `false` to opt out. Use `buildCodegraphArtifactWithSession()` when a host already has a session and wants SQLite, graph JSON, report, questions, and manifest outputs from the same snapshot. `createCodegraphMcpHandlers()` exposes the same primitives without starting stdio, which is useful for tests or host applications: +`createAgentSession()` keeps one in-process project snapshot warm for repeated orient, search, explain, packet, artifact, and MCP calls. It uses incremental indexing with disk cache by default, auto-enables native workers for large cold builds, and carries forward top-level analysis metadata from the build report. +Session callers can use `freshness: { policy: "check" | "auto" | "manual" }` plus `checkFreshness()` to detect file edits before reusing a warm snapshot. `check` reports stale state without invalidating, `auto` invalidates for bounded changes, and stale results include `changedFileCount`, `omittedChangedFileCount`, `reason`, and a bounded changed-file sample. +Set `buildOptions.useNativeWorkers` to `false` to opt out. Use `buildCodegraphArtifactWithSession()` when a host already has a session and wants SQLite, graph JSON, report, questions, and manifest outputs from the same snapshot. `createCodegraphMcpHandlers()` exposes the same primitives without starting stdio, which is useful for tests or host applications: ```ts import { createCodegraphMcpHandlers } from "@lzehrung/codegraph"; @@ -178,11 +180,13 @@ const orient = await handlers.orient({ includeRoots: ["src"], budget: "small" }) const packet = await handlers.packet_get({ target: orient.focus.find((entry) => entry.file)!.file! }); const refs = await handlers.refs({ handle: search.results[0]!.handle }); const rows = await handlers.query_sqlite({ query: "select path from files", limit: 5 }); -console.log(packet.kind, refs.references, rows.rows); +console.log(search.freshness.state); +console.log(packet.kind, refs.references, rows.rows, rows.freshness.state); ``` -`serveCodegraphMcp()` starts the stdio server used by `codegraph mcp serve`. MCP is an agent ergonomics and cache layer over the same analysis engine, not a separate indexer. MCP file and artifact paths are confined after realpath resolution. `query_sqlite` is read-only and row- and byte-bounded; `artifact_build` is disabled by default and requires `readOnly: false` or CLI `--allow-build`. -MCP `orient` and `packet_get` calls use the server-configured root; they do not accept per-request root overrides. +`serveCodegraphMcp()` starts the stdio server used by `codegraph mcp serve`. MCP is an agent ergonomics and cache layer over the same analysis engine, not a separate indexer. MCP file and artifact paths are confined after realpath resolution. +`query_sqlite` is read-only and row- and byte-bounded. It returns freshness metadata for fresh artifact reads, refreshes Codegraph-owned SQLite artifacts after small edits when write access is enabled, and rejects stale artifact queries it cannot refresh safely. +`artifact_build` is disabled by default and requires `readOnly: false` or CLI `--allow-build`; it refuses to write outputs from a stale MCP index until `refresh_index` succeeds. MCP `orient` and `packet_get` calls use the server-configured root; they do not accept per-request root overrides. See [MCP server](./mcp.md) for CLI server setup and client configuration examples. diff --git a/docs/mcp.md b/docs/mcp.md index 800072ed..24e5500b 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -44,12 +44,13 @@ The server exposes the same bounded primitives as the CLI and library session la - `refs`: references by handle or file position. - `deps`, `rdeps`, `path`: dependency navigation. - `impact`, `review`: git-range risk and review context. -- `query_sqlite`: bounded read-only SQLite artifact query. +- `query_sqlite`: bounded read-only SQLite artifact query with freshness metadata. - `refresh_index`: invalidate the in-memory session and optionally rebuild the base or symbol snapshot. - `artifact_build`: artifact creation, available only with write access enabled. MCP keeps one Codegraph session warm for the configured root. That makes follow-up calls cheaper than separate CLI invocations. Startup is lazy unless `--warmup` or `--warmup-symbols` is passed. -Use `refresh_index` after changing files while the server is running, or when you need a fresh cache-backed snapshot without restarting the MCP process. +Before index-backed tool calls, MCP checks whether discovered files changed since the warm snapshot. Small changes refresh the session automatically, and responses include `freshness.state` as `fresh`, `refreshed`, or `stale`; stale responses also include `changedFileCount`, `omittedChangedFileCount`, and a bounded changed-file sample. +Use `refresh_index` when you need to force a rebuild, reset SQLite artifact state, or refresh after a change burst that exceeds the automatic refresh limits. `query_sqlite` refreshes Codegraph-owned SQLite artifacts after small edits when write access is enabled; otherwise it refuses to serve stale artifact rows. `artifact_build` refuses to write outputs from a stale MCP index; run `refresh_index` first after large change bursts. Tool schemas are flat JSON objects for broad client compatibility; argument combinations such as `refs` handle-vs-position mode are validated by the server. ## Safety @@ -57,8 +58,8 @@ Tool schemas are flat JSON objects for broad client compatibility; argument comb - File and artifact paths are confined to `--root` after realpath resolution. - Tool calls do not accept per-request root overrides. - Tools are read-only by default. -- `artifact_build` requires `--allow-build`. -- `query_sqlite` rejects mutating SQL, recursive queries, and synthetic payload functions. +- `artifact_build` requires `--allow-build` and a fresh or auto-refreshed MCP index. +- `query_sqlite` rejects mutating SQL, recursive queries, synthetic payload functions, and stale artifact queries it cannot refresh safely. - SQLite responses are row- and byte-bounded. ## Client Configuration Examples @@ -195,9 +196,10 @@ When Codegraph MCP tools are available to an agent: 1. Start with `orient`. 2. Use `search` to find anchors. 3. Use `packet_get`, `refs`, `goto`, `deps`, `rdeps`, or `path` for focused follow-up. -4. Use `impact` and `review` for git-range risk analysis. -5. Use `query_sqlite` only for read-only artifact inspection. -6. Use `refresh_index` after changing files while a long-running server is active. -7. Use `artifact_build` only when write access was intentionally enabled. +4. Check `freshness` on MCP responses after edits; `refreshed` means the answer used an updated snapshot, and `stale` includes a reason plus a bounded changed-file sample. +5. Use `impact` and `review` for git-range risk analysis. +6. Use `query_sqlite` only for read-only artifact inspection; rebuild the artifact when it reports stale state. +7. Use `refresh_index` when you need an explicit rebuild. +8. Use `artifact_build` only when write access was intentionally enabled. Fall back to CLI commands when MCP tools are unavailable. diff --git a/src/agent.ts b/src/agent.ts index 65b31881..06e72765 100644 --- a/src/agent.ts +++ b/src/agent.ts @@ -1,5 +1,12 @@ export { createAgentSession } from "./agent/session.js"; -export type { AgentProjectSnapshot, AgentSession, AgentSessionOptions } from "./agent/session.js"; +export type { + AgentFreshnessPolicy, + AgentFreshnessResult, + AgentProjectSnapshot, + AgentSession, + AgentSessionFreshnessOptions, + AgentSessionOptions, +} from "./agent/session.js"; export { orientCodegraph } from "./agent/orient.js"; export type { AgentModuleSummary, diff --git a/src/agent/artifact.ts b/src/agent/artifact.ts index 606b0a82..a90295c9 100644 --- a/src/agent/artifact.ts +++ b/src/agent/artifact.ts @@ -114,11 +114,23 @@ export async function buildCodegraphArtifactWithSession( const artifacts: CodegraphArtifactBuildResult["artifacts"] = {}; if (selected.sqlite) { + if (!snapshot.fileSignatures) { + throw new Error("SQLite artifact freshness signatures are unavailable."); + } + const fileSignatures: Array<{ path: string; size: number; mtimeMs: number }> = []; + for (const file of snapshot.fileGraph.nodes) { + const signature = snapshot.fileSignatures.get(file); + if (!signature) { + throw new Error(`SQLite artifact freshness signature is missing for ${file}.`); + } + fileSignatures.push({ path: signature.file, size: signature.size, mtimeMs: signature.mtimeMs }); + } const outputPath = path.join(outDir, SQLITE_FILE); await writeGraphSqlite({ fileGraph: snapshot.fileGraph, symbolGraph: snapshot.symbolGraph, outputPath, + fileSignatures, }); artifacts.sqlite = SQLITE_FILE; } diff --git a/src/agent/session.ts b/src/agent/session.ts index 67d3d9dc..60eedba3 100644 --- a/src/agent/session.ts +++ b/src/agent/session.ts @@ -1,9 +1,13 @@ +import fsp from "node:fs/promises"; +import path from "node:path"; import { buildProjectIndexIncremental } from "../indexer/build-index.js"; import type { BuildOptions, BuildReport, ProjectIndex } from "../indexer/types.js"; import { buildSymbolGraphDetailed } from "../graphs/symbol-graph-detailed.js"; import { type SymbolGraph } from "../graphs/symbol-graph.js"; import type { Graph } from "../types.js"; import { listProjectFiles, type ProjectFileDiscoveryOptions } from "../util/projectFiles.js"; +import { mapLimit } from "../util/concurrency.js"; +import { normalizePath, toProjectDisplayPath } from "../util/paths.js"; import { hasDiscoveryOptions, loadCodegraphConfig, mergeDiscoveryOptions } from "../config.js"; import { createAgentFileLookup } from "./normalize.js"; import { summarizeAnalysis, type AnalysisSummary } from "../analysisSummary.js"; @@ -17,33 +21,157 @@ export type AgentProjectSnapshot = { symbolGraph: SymbolGraph; buildReport?: BuildReport; analysis: AnalysisSummary; + fileSignatures?: ReadonlyMap; }; export type AgentLoadProjectOptions = { symbolGraph?: "eager" | "skip"; }; +export type AgentFreshnessPolicy = "manual" | "check" | "auto"; + +export type AgentFreshnessResult = + | { state: "fresh" } + | { state: "refreshed"; changedFiles: string[] } + | { + state: "stale"; + changedFiles: string[]; + changedFileCount: number; + omittedChangedFileCount: number; + reason: string; + }; + +export type AgentSessionFreshnessOptions = { + policy?: AgentFreshnessPolicy; + maxAutoRefreshFiles?: number; + maxAutoRefreshBytes?: number; +}; + export type AgentSessionOptions = { root: string; discovery?: ProjectFileDiscoveryOptions; buildOptions?: BuildOptions; useConfig?: boolean; + freshness?: AgentSessionFreshnessOptions; }; export type AgentSession = { root?: string; + discoverFiles?: () => Promise; listFiles?: () => Promise; loadProject: (loadOptions?: AgentLoadProjectOptions) => Promise; + checkFreshness?: () => Promise; invalidate: () => void; }; -type AgentProjectBaseSnapshot = Omit; - const EMPTY_SYMBOL_GRAPH: SymbolGraph = { nodes: new Map(), edges: [], }; const AGENT_NATIVE_WORKER_AUTO_FILE_THRESHOLD = 250; +const DEFAULT_MAX_AUTO_REFRESH_FILES = 50; +const DEFAULT_MAX_AUTO_REFRESH_BYTES = 2_000_000; +const DEFAULT_MAX_FRESHNESS_CHANGED_FILES = 25; + +type AgentProjectBaseSnapshot = Omit; + +export type AgentFileSignature = { + file: string; + size: number; + mtimeMs: number; +}; + +type AgentDiscoverySettings = { + discoveryOptions?: ProjectFileDiscoveryOptions; +}; + +type AgentFreshnessDiff = { + changedFiles: string[]; + changedBytes: number; +}; + +async function resolveAgentDiscoverySettings(options: AgentSessionOptions): Promise { + const useConfig = options.useConfig ?? true; + const config = useConfig ? await loadCodegraphConfig(options.root) : {}; + const optionDiscovery = mergeDiscoveryOptions(options.buildOptions?.discovery, options.discovery); + const discovery = mergeDiscoveryOptions(config.discovery, optionDiscovery); + const discoveryOptions = hasDiscoveryOptions(discovery) + ? { ...discovery, globRoot: discovery.globRoot ?? options.root } + : undefined; + return discoveryOptions ? { discoveryOptions } : {}; +} + +export async function listAgentSessionFiles(options: AgentSessionOptions): Promise { + const { discoveryOptions } = await resolveAgentDiscoverySettings(options); + return await listProjectFiles(options.root, undefined, discoveryOptions); +} + +function isMissingStatRace(error: unknown): boolean { + if (!(error instanceof Error)) return false; + if (!("code" in error)) return false; + return error.code === "ENOENT" || error.code === "ENOTDIR"; +} + +function summarizeChangedFiles(files: readonly string[]): { + changedFiles: string[]; + changedFileCount: number; + omittedChangedFileCount: number; +} { + const changedFiles = files.slice(0, DEFAULT_MAX_FRESHNESS_CHANGED_FILES); + return { + changedFiles, + changedFileCount: files.length, + omittedChangedFileCount: Math.max(0, files.length - changedFiles.length), + }; +} + +const FILE_SIGNATURE_STAT_CONCURRENCY = 64; + +async function collectAgentFileSignatures(files: readonly string[]): Promise> { + const signatures = new Map(); + await mapLimit([...files], FILE_SIGNATURE_STAT_CONCURRENCY, async (file) => { + const resolvedFile = normalizePath(path.resolve(file)); + try { + const stat = await fsp.stat(resolvedFile); + if (!stat.isFile()) return; + signatures.set(resolvedFile, { + file: resolvedFile, + size: stat.size, + mtimeMs: stat.mtimeMs, + }); + } catch (error) { + if (isMissingStatRace(error)) return; + throw new Error(`Unable to verify freshness for ${resolvedFile}`, { cause: error }); + } + }); + return signatures; +} + +function diffAgentFileSignatures( + previous: ReadonlyMap, + current: ReadonlyMap, +): AgentFreshnessDiff { + const changedFiles: string[] = []; + let changedBytes = 0; + for (const [file, currentSignature] of current.entries()) { + const previousSignature = previous.get(file); + if ( + !previousSignature || + previousSignature.size !== currentSignature.size || + previousSignature.mtimeMs !== currentSignature.mtimeMs + ) { + changedFiles.push(file); + changedBytes += currentSignature.size; + } + } + for (const [file, previousSignature] of previous.entries()) { + if (current.has(file)) continue; + changedFiles.push(file); + changedBytes += previousSignature.size; + } + changedFiles.sort(); + return { changedFiles, changedBytes }; +} export function createAgentSession(options: AgentSessionOptions): AgentSession { let cachedFiles: Promise | undefined; @@ -51,20 +179,20 @@ export function createAgentSession(options: AgentSessionOptions): AgentSession { let cachedSymbolGraph: Promise | undefined; let cachedEagerSnapshot: Promise | undefined; let cachedSkippedSnapshot: Promise | undefined; + let cachedFileSignatures: Map | undefined; + + const invalidate = (): void => { + cachedFiles = undefined; + cachedBase = undefined; + cachedSymbolGraph = undefined; + cachedEagerSnapshot = undefined; + cachedSkippedSnapshot = undefined; + cachedFileSignatures = undefined; + }; const loadFiles = async (): Promise => { if (cachedFiles) return cachedFiles; - const loadPromise = (async () => { - const useConfig = options.useConfig ?? true; - const config = useConfig ? await loadCodegraphConfig(options.root) : {}; - const optionDiscovery = mergeDiscoveryOptions(options.buildOptions?.discovery, options.discovery); - const discovery = mergeDiscoveryOptions(config.discovery, optionDiscovery); - const discoveryOptions = hasDiscoveryOptions(discovery) - ? { ...discovery, globRoot: discovery.globRoot ?? options.root } - : undefined; - - return await listProjectFiles(options.root, undefined, discoveryOptions); - })(); + const loadPromise = listAgentSessionFiles(options); cachedFiles = loadPromise; loadPromise.catch(() => { if (cachedFiles === loadPromise) cachedFiles = undefined; @@ -75,14 +203,9 @@ export function createAgentSession(options: AgentSessionOptions): AgentSession { const loadBase = async (): Promise => { if (cachedBase) return cachedBase; const loadPromise = (async () => { - const useConfig = options.useConfig ?? true; - const config = useConfig ? await loadCodegraphConfig(options.root) : {}; - const optionDiscovery = mergeDiscoveryOptions(options.buildOptions?.discovery, options.discovery); - const discovery = mergeDiscoveryOptions(config.discovery, optionDiscovery); - const discoveryOptions = hasDiscoveryOptions(discovery) - ? { ...discovery, globRoot: discovery.globRoot ?? options.root } - : undefined; + const { discoveryOptions } = await resolveAgentDiscoverySettings(options); const files = await loadFiles(); + cachedFileSignatures = await collectAgentFileSignatures(files); const buildOptions: BuildOptions & { files: string[] } = { ...options.buildOptions, cache: options.buildOptions?.cache ?? "disk", @@ -107,6 +230,7 @@ export function createAgentSession(options: AgentSessionOptions): AgentSession { fileLookup: createAgentFileLookup(files), index, fileGraph, + fileSignatures: cachedFileSignatures, buildReport, analysis: summarizeAnalysis({ index, report: buildReport }), }; @@ -151,16 +275,54 @@ export function createAgentSession(options: AgentSessionOptions): AgentSession { return await cachedEagerSnapshot; }; + const checkFreshness = async (): Promise => { + const policy = options.freshness?.policy ?? "check"; + if (policy === "manual") return { state: "fresh" }; + if (!cachedBase || !cachedFileSignatures) return { state: "fresh" }; + await cachedBase; + + const { discoveryOptions } = await resolveAgentDiscoverySettings(options); + const currentFiles = await listProjectFiles(options.root, undefined, discoveryOptions); + const currentSignatures = await collectAgentFileSignatures(currentFiles); + const diff = diffAgentFileSignatures(cachedFileSignatures, currentSignatures); + if (!diff.changedFiles.length) return { state: "fresh" }; + + const changedFiles = diff.changedFiles.map((file) => toProjectDisplayPath(options.root, file)); + if (policy === "check") { + return { + state: "stale", + ...summarizeChangedFiles(changedFiles), + reason: "session snapshot is older than files on disk", + }; + } + + const maxFiles = options.freshness?.maxAutoRefreshFiles ?? DEFAULT_MAX_AUTO_REFRESH_FILES; + const maxBytes = options.freshness?.maxAutoRefreshBytes ?? DEFAULT_MAX_AUTO_REFRESH_BYTES; + if (diff.changedFiles.length > maxFiles) { + return { + state: "stale", + ...summarizeChangedFiles(changedFiles), + reason: `changed file count exceeds ${maxFiles}`, + }; + } + if (diff.changedBytes > maxBytes) { + return { + state: "stale", + ...summarizeChangedFiles(changedFiles), + reason: `changed byte count exceeds ${maxBytes}`, + }; + } + + invalidate(); + return { state: "refreshed", changedFiles }; + }; + return { root: options.root, + discoverFiles: () => listAgentSessionFiles(options), listFiles: loadFiles, loadProject, - invalidate: () => { - cachedFiles = undefined; - cachedBase = undefined; - cachedSymbolGraph = undefined; - cachedEagerSnapshot = undefined; - cachedSkippedSnapshot = undefined; - }, + checkFreshness, + invalidate, }; } diff --git a/src/index.ts b/src/index.ts index 7fafd763..f70c8f33 100644 --- a/src/index.ts +++ b/src/index.ts @@ -226,7 +226,14 @@ export { /** Agent project snapshots and cached service-layer helpers. */ export { createAgentSession } from "./agent/session.js"; -export type { AgentProjectSnapshot, AgentSession, AgentSessionOptions } from "./agent/session.js"; +export type { + AgentFreshnessPolicy, + AgentFreshnessResult, + AgentProjectSnapshot, + AgentSession, + AgentSessionFreshnessOptions, + AgentSessionOptions, +} from "./agent/session.js"; /** Agent first-turn orientation packets with file-path follow-up targets. */ export { orientCodegraph } from "./agent/orient.js"; @@ -309,6 +316,7 @@ export { updateGraphSqlite, queryGraphSqlite, queryGraphSqliteRaw, + SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY, type SqliteGraphOptions, type SqliteGraphUpdateOptions, type GraphQueryResult, diff --git a/src/mcp/server.ts b/src/mcp/server.ts index 222ed7a3..9df1e466 100644 --- a/src/mcp/server.ts +++ b/src/mcp/server.ts @@ -24,11 +24,13 @@ import type { AgentSearchMode, AgentSearchResponse } from "../agent/search.js"; import { getDependencies, getReverseDependencies, getShortestPath, type DependencyNode } from "../graphs/queries.js"; import { findReferences, goToDefinition } from "../indexer/navigation.js"; import { buildReviewReport, type ReviewDepth, type ReviewReport } from "../review.js"; -import { queryGraphSqliteRaw, type RawSqlResult } from "../sqlite.js"; +import { SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY, queryGraphSqliteRaw, type RawSqlResult } from "../sqlite.js"; +import { isPlainRecord } from "../util/guards.js"; import { toProjectDisplayPath } from "../util/paths.js"; -import { createAgentSession } from "../agent/session.js"; -import type { AgentSession } from "../agent/session.js"; -import type { BuildOptions } from "../indexer/types.js"; +import { createAgentSession, listAgentSessionFiles } from "../agent/session.js"; +import { mapLimit } from "../util/concurrency.js"; +import type { AgentFreshnessResult, AgentProjectSnapshot, AgentSession } from "../agent/session.js"; +import type { BuildOptions, GoToResult } from "../indexer/types.js"; import { assertMcpSqliteQueryResourceBounded, boundRawSqlResult, @@ -98,6 +100,8 @@ export type CodegraphMcpHttpServer = CodegraphMcpHttpServerInfo & { close: () => Promise; }; +export type CodegraphMcpFreshResult = T & { freshness: AgentFreshnessResult }; + export type CodegraphMcpHandlers = { search: (request: { query: string; @@ -105,45 +109,45 @@ export type CodegraphMcpHandlers = { from?: string | undefined; depth?: number | undefined; limit?: number | undefined; - }) => Promise; + }) => Promise>; orient: (request: { includeRoots?: string[] | undefined; budget?: AgentOrientBudget | undefined; - }) => Promise; + }) => Promise>; packet_get: (request: { target: string; maxSymbols?: number | undefined; maxSnippets?: number | undefined; maxDuplicates?: number | undefined; - }) => Promise; + }) => Promise>; get_file: (request: { file: string; maxBytes?: number | undefined; - }) => Promise<{ file: string; text: string; truncated: boolean }>; - get_symbol: (request: { handle: string }) => Promise; - goto: (request: { - file: string; - line: number; - column: number; - }) => Promise>>; + }) => Promise>; + get_symbol: (request: { handle: string }) => Promise>; + goto: (request: { file: string; line: number; column: number }) => Promise>; refs: ( request: | { handle: string; limit?: number | undefined } | { file: string; line: number; column: number; limit?: number | undefined }, - ) => Promise<{ references: AgentExplanationReference[] }>; + ) => Promise>; deps: (request: { file: string; depth?: number | undefined; limit?: number | undefined; - }) => Promise<{ dependencies: Array<{ file: string; depth: number }> }>; + }) => Promise }>>; rdeps: (request: { file: string; depth?: number | undefined; limit?: number | undefined; - }) => Promise<{ reverseDependencies: Array<{ file: string; depth: number }> }>; - path: (request: { from: string; to: string }) => Promise<{ path: string[] | null }>; - impact: (request: { base: string; head: string }) => Promise; - review: (request: { base: string; head: string; reviewDepth?: ReviewDepth | undefined }) => Promise; + }) => Promise }>>; + path: (request: { from: string; to: string }) => Promise>; + impact: (request: { base: string; head: string }) => Promise>; + review: (request: { + base: string; + head: string; + reviewDepth?: ReviewDepth | undefined; + }) => Promise>; refresh_index: (request: { warmup?: CodegraphMcpWarmupMode | undefined }) => Promise<{ refreshed: true; warmup: CodegraphMcpWarmupMode; @@ -152,7 +156,7 @@ export type CodegraphMcpHandlers = { query: string; params?: Array | undefined; limit?: number | undefined; - }) => Promise; + }) => Promise>; artifact_build: (request: { outDir?: string | undefined; sqlite?: boolean | undefined; @@ -160,7 +164,7 @@ export type CodegraphMcpHandlers = { report?: boolean | undefined; questions?: boolean | undefined; force?: boolean | undefined; - }) => Promise; + }) => Promise>; }; type McpDependencyRequest = { @@ -174,6 +178,15 @@ type McpDependencyEntry = { depth: number; }; +type SqliteArtifactFileSignature = { + path: string; + size: number; + mtimeMs: number; +}; + +const MAX_MCP_FRESHNESS_CHANGED_FILES = 25; +const SQLITE_ARTIFACT_STAT_CONCURRENCY = 64; + function assertMcpSessionOptions(options: CodegraphMcpHandlerOptions): void { if (options.session !== undefined && options.buildOptions !== undefined) { throw new Error("MCP server options cannot combine a prebuilt session with buildOptions."); @@ -184,14 +197,18 @@ function createCodegraphMcpSession(options: CodegraphMcpHandlerOptions, root: st assertMcpSessionOptions(options); return ( options.session ?? - createAgentSession({ root, ...(options.buildOptions ? { buildOptions: options.buildOptions } : {}) }) + createAgentSession({ + root, + ...(options.buildOptions ? { buildOptions: options.buildOptions } : {}), + freshness: { policy: "auto" }, + }) ); } function startCodegraphMcpWarmup( session: AgentSession, warmup: CodegraphMcpWarmupMode | undefined, -): Promise>> | undefined { +): Promise | undefined { if (warmup === "base") { return session.loadProject({ symbolGraph: "skip" }); } @@ -232,17 +249,191 @@ function createCodegraphMcpHandlersForSession( const configuredSqlitePath = options.artifactPath ? resolveArtifactSqlitePathCandidate(root, options.artifactPath) : undefined; + const configuredSqliteOutDir = configuredSqlitePath ? path.dirname(configuredSqlitePath) : undefined; + const configuredSqliteCanRefresh = options.artifactPath ? !/\.(sqlite|db)$/i.test(options.artifactPath) : false; let sqlitePath = configuredSqlitePath; + let sqliteOutDir = configuredSqliteOutDir; + let sqliteCanRefresh = configuredSqliteCanRefresh; const relative = (file: string): string => toProjectDisplayPath(root, file); const boundedLimit = (limit: number | undefined, fallback: number, max: number): number => { if (typeof limit !== "number" || !Number.isFinite(limit)) return fallback; return Math.min(max, Math.max(0, Math.floor(limit))); }; + const staleFreshness = (files: readonly string[], reason: string): AgentFreshnessResult => { + const changedFiles = [...files].sort(); + const boundedChangedFiles = changedFiles.slice(0, MAX_MCP_FRESHNESS_CHANGED_FILES); + return { + state: "stale", + changedFiles: boundedChangedFiles, + changedFileCount: changedFiles.length, + omittedChangedFileCount: Math.max(0, changedFiles.length - boundedChangedFiles.length), + reason, + }; + }; + const checkMcpFreshness = async (): Promise => { + if (session.checkFreshness) return await session.checkFreshness(); + return { state: "fresh" }; + }; + const withFreshness = async ( + run: () => Promise, + ): Promise => { + const freshness = await checkMcpFreshness(); + const result = await run(); + return { ...result, freshness }; + }; + const formatSqliteFreshnessError = (freshness: AgentFreshnessResult): string => { + if (freshness.state === "fresh") return "SQLite artifact freshness check unexpectedly failed."; + const reason = freshness.state === "stale" ? freshness.reason : "workspace changed after artifact build"; + const changed = freshness.changedFiles.length ? ` Changed files: ${freshness.changedFiles.join(", ")}.` : ""; + const omitted = + freshness.state === "stale" && freshness.omittedChangedFileCount + ? ` Omitted changed files: ${freshness.omittedChangedFileCount}.` + : ""; + let action = "run artifact_build"; + if (readOnly) { + action = "rebuild the artifact with write access enabled"; + } + if (freshness.state === "stale") { + action = `run refresh_index, then ${action}`; + } + return `SQLite artifact is stale; ${action} before query_sqlite. ${reason}.${changed}${omitted}`; + }; + const canRefreshSqliteArtifact = (): boolean => { + if (!sqlitePath || !sqliteOutDir || readOnly || !sqliteCanRefresh) return false; + return path.basename(sqlitePath) === "codegraph.sqlite"; + }; + const rebuildSqliteArtifactForQuery = async (): Promise => { + if (!sqliteOutDir) throw new Error("SQLite artifact output directory is unavailable."); + const outDir = await assertWritableDirectoryRealPathWithinRoot( + await realRoot, + root, + sqliteOutDir, + "Artifact output directory", + ); + const result = await buildCodegraphArtifactWithSession(session, { + root, + outDir, + filterOutDir: outDir, + sqlite: true, + force: true, + }); + const sqliteArtifact = result.artifacts.sqlite; + if (!sqliteArtifact) throw new Error("SQLite artifact refresh did not produce a SQLite file."); + sqlitePath = path.join(result.outDir, sqliteArtifact); + sqliteOutDir = result.outDir; + }; + const refreshSqliteArtifactForQuery = async ( + freshness: AgentFreshnessResult, + refreshOptions?: { allowStaleRebuild?: boolean }, + ): Promise => { + if (freshness.state === "fresh") return freshness; + if (freshness.state === "stale" && !refreshOptions?.allowStaleRebuild) { + throw new Error(formatSqliteFreshnessError(freshness)); + } + if (!canRefreshSqliteArtifact()) throw new Error(formatSqliteFreshnessError(freshness)); + await rebuildSqliteArtifactForQuery(); + return { state: "refreshed", changedFiles: freshness.changedFiles }; + }; + const readSqliteArtifactSignatures = async ( + realSqlitePath: string, + ): Promise => { + const result = await queryGraphSqliteRaw( + realSqlitePath, + "SELECT value FROM graph_metadata WHERE key = ?;", + [SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY], + { maxRows: 1 }, + ); + const rawValue = result.rows[0]?.[0]; + if (typeof rawValue !== "string") return null; + let parsed: unknown; + try { + parsed = JSON.parse(rawValue); + } catch { + return null; + } + if (!Array.isArray(parsed)) return null; + const signatures: SqliteArtifactFileSignature[] = []; + for (const value of parsed) { + if (!isPlainRecord(value)) continue; + if (typeof value.path !== "string") continue; + if (typeof value.size !== "number" || typeof value.mtimeMs !== "number") continue; + signatures.push({ path: value.path, size: value.size, mtimeMs: value.mtimeMs }); + } + return signatures; + }; + const isFileInsideDirectory = (file: string, directory: string): boolean => { + const relativeFile = path.relative(directory, file); + if (!relativeFile) return true; + return !relativeFile.startsWith("..") && !path.isAbsolute(relativeFile); + }; + const collectCurrentSqliteArtifactSignatures = async (): Promise> => { + const outputDirectories: string[] = []; + if (sqliteOutDir) { + outputDirectories.push(sqliteOutDir); + const lexicalOutDir = path.resolve(root, path.relative(await realRoot, sqliteOutDir)); + outputDirectories.push(lexicalOutDir); + } + let discoveredFiles: string[]; + if (session.discoverFiles) { + discoveredFiles = await session.discoverFiles(); + } else if (options.session) { + throw new Error("MCP session does not expose live file discovery for SQLite freshness."); + } else { + discoveredFiles = await listAgentSessionFiles({ + root, + ...(options.buildOptions ? { buildOptions: options.buildOptions } : {}), + }); + } + const currentFiles = discoveredFiles.filter( + (file) => !outputDirectories.some((directory) => isFileInsideDirectory(file, directory)), + ); + const signatures = new Map(); + await mapLimit(currentFiles, SQLITE_ARTIFACT_STAT_CONCURRENCY, async (file) => { + try { + const stat = await fs.stat(file); + if (!stat.isFile()) return; + signatures.set(relative(file), { path: file, size: stat.size, mtimeMs: stat.mtimeMs }); + } catch (error) { + if (error instanceof Error && "code" in error && (error.code === "ENOENT" || error.code === "ENOTDIR")) { + return; + } + throw error; + } + }); + return signatures; + }; + const checkSqliteArtifactFreshness = async (realSqlitePath: string): Promise => { + const storedSignatures = await readSqliteArtifactSignatures(realSqlitePath); + if (!storedSignatures) return staleFreshness([], "SQLite artifact has no freshness baseline"); + const storedByFile = new Map(); + for (const signature of storedSignatures) { + storedByFile.set(relative(signature.path), signature); + } + const currentByFile = await collectCurrentSqliteArtifactSignatures(); + const changedFiles: string[] = []; + for (const [file, currentSignature] of currentByFile.entries()) { + const storedSignature = storedByFile.get(file); + if (!storedSignature) { + changedFiles.push(file); + continue; + } + if (storedSignature.size !== currentSignature.size || storedSignature.mtimeMs !== currentSignature.mtimeMs) { + changedFiles.push(file); + } + } + for (const file of storedByFile.keys()) { + if (currentByFile.has(file)) continue; + changedFiles.push(file); + } + if (!changedFiles.length) return { state: "fresh" }; + return staleFreshness(changedFiles, "SQLite artifact is older than files on disk"); + }; + const collectMcpDependencyEntries = async ( request: McpDependencyRequest, collectEntries: ( - graph: Awaited>["fileGraph"], + graph: AgentProjectSnapshot["fileGraph"], file: string, options: { depth?: number; limit: number }, ) => DependencyNode[], @@ -264,32 +455,44 @@ function createCodegraphMcpHandlersForSession( return { search: async (request) => - await searchCodegraphWithSession(session, { - root, - query: request.query, - ...(request.mode !== undefined ? { mode: request.mode } : {}), - ...(request.from !== undefined ? { from: request.from } : {}), - ...(request.depth !== undefined ? { depth: request.depth } : {}), - ...(request.limit !== undefined ? { limit: request.limit } : {}), - }), + await withFreshness( + async () => + await searchCodegraphWithSession(session, { + root, + query: request.query, + ...(request.mode !== undefined ? { mode: request.mode } : {}), + ...(request.from !== undefined ? { from: request.from } : {}), + ...(request.depth !== undefined ? { depth: request.depth } : {}), + ...(request.limit !== undefined ? { limit: request.limit } : {}), + }), + ), orient: async (request) => - await orientCodegraphWithSession(session, { - root, - ...(request.includeRoots !== undefined ? { includeRoots: request.includeRoots } : {}), - ...(request.budget !== undefined ? { budget: request.budget } : {}), - }), + await withFreshness( + async () => + await orientCodegraphWithSession(session, { + root, + ...(request.includeRoots !== undefined ? { includeRoots: request.includeRoots } : {}), + ...(request.budget !== undefined ? { budget: request.budget } : {}), + }), + ), packet_get: async (request) => - await getCodegraphPacketWithSession(session, { - root, - target: request.target, - ...(request.maxSymbols !== undefined ? { maxSymbols: request.maxSymbols } : {}), - ...(request.maxSnippets !== undefined ? { maxSnippets: request.maxSnippets } : {}), - ...(request.maxDuplicates !== undefined ? { maxDuplicates: request.maxDuplicates } : {}), - }), + await withFreshness( + async () => + await getCodegraphPacketWithSession(session, { + root, + target: request.target, + ...(request.maxSymbols !== undefined ? { maxSymbols: request.maxSymbols } : {}), + ...(request.maxSnippets !== undefined ? { maxSnippets: request.maxSnippets } : {}), + ...(request.maxDuplicates !== undefined ? { maxDuplicates: request.maxDuplicates } : {}), + }), + ), get_file: async (request) => { + // get_file returns live bytes read directly from disk and never consults the indexed + // session, so it must not trigger a workspace-wide freshness scan or rebuild. The bytes + // are always current, so report fresh without re-stating the project. const resolvedFile = await resolveReadableFile(await realRoot, root, request.file); const maxBytes = boundedLimit(request.maxBytes, DEFAULT_FILE_BYTES, MAX_FILE_BYTES); const read = await readFilePrefix(resolvedFile.realPath, maxBytes); @@ -297,125 +500,153 @@ function createCodegraphMcpHandlersForSession( file: resolvedFile.displayPath, text: read.text, truncated: read.truncated, + freshness: { state: "fresh" }, }; }, - get_symbol: async (request) => { - const explanation = await explainCodegraphTargetWithSession(session, { root, target: request.handle }); - return explanation.target; - }, - - goto: async (request) => { - const snapshot = await session.loadProject({ symbolGraph: "skip" }); - return await goToDefinition(snapshot.index, { - file: await resolveProjectFile(await realRoot, root, request.file), - line: request.line, - column: request.column, - }); - }, - - refs: async (request) => { - const handle = "handle" in request ? request.handle : undefined; - const file = "file" in request ? request.file : undefined; - const line = "line" in request ? request.line : undefined; - const column = "column" in request ? request.column : undefined; - const hasAnyPosition = file !== undefined || line !== undefined || column !== undefined; - const hasCompletePosition = file !== undefined && line !== undefined && column !== undefined; - if (handle !== undefined && hasAnyPosition) { - throw new Error("refs requires either handle or file, line, and column."); - } - if (handle === undefined && !hasCompletePosition) { - throw new Error("refs requires either handle or file, line, and column."); - } + get_symbol: async (request) => + await withFreshness(async () => { + const explanation = await explainCodegraphTargetWithSession(session, { root, target: request.handle }); + return explanation.target; + }), - if (handle !== undefined) { - const explanation = await explainCodegraphTargetWithSession(session, { - root, - target: handle, - maxReferences: boundedLimit(request.limit, DEFAULT_MCP_COLLECTION_LIMIT, MAX_MCP_COLLECTION_LIMIT), + goto: async (request) => + await withFreshness(async () => { + const snapshot = await session.loadProject({ symbolGraph: "skip" }); + return await goToDefinition(snapshot.index, { + file: await resolveProjectFile(await realRoot, root, request.file), + line: request.line, + column: request.column, }); - return { references: explanation.references }; - } - if (file === undefined || line === undefined || column === undefined) { - throw new Error("refs requires either handle or file, line, and column."); - } + }), - const snapshot = await session.loadProject({ symbolGraph: "skip" }); - const referenceOptions = { - maxReferences: boundedLimit(request.limit, DEFAULT_MCP_COLLECTION_LIMIT, MAX_MCP_COLLECTION_LIMIT), - }; - const result = await findReferences( - snapshot.index, - { - file: await resolveProjectFile(await realRoot, root, file), - line, - column, - }, - referenceOptions, - ); - if (result.status !== "ok") return { references: [] }; - return { - references: result.references.map((reference) => ({ - file: relative(reference.file), - range: reference.range, - })), - }; - }, + refs: async (request) => + await withFreshness(async () => { + const handle = "handle" in request ? request.handle : undefined; + const file = "file" in request ? request.file : undefined; + const line = "line" in request ? request.line : undefined; + const column = "column" in request ? request.column : undefined; + const hasAnyPosition = file !== undefined || line !== undefined || column !== undefined; + const hasCompletePosition = file !== undefined && line !== undefined && column !== undefined; + if (handle !== undefined && hasAnyPosition) { + throw new Error("refs requires either handle or file, line, and column."); + } + if (handle === undefined && !hasCompletePosition) { + throw new Error("refs requires either handle or file, line, and column."); + } + + if (handle !== undefined) { + const explanation = await explainCodegraphTargetWithSession(session, { + root, + target: handle, + maxReferences: boundedLimit(request.limit, DEFAULT_MCP_COLLECTION_LIMIT, MAX_MCP_COLLECTION_LIMIT), + }); + return { references: explanation.references }; + } + if (file === undefined || line === undefined || column === undefined) { + throw new Error("refs requires either handle or file, line, and column."); + } + + const snapshot = await session.loadProject({ symbolGraph: "skip" }); + const referenceOptions = { + maxReferences: boundedLimit(request.limit, DEFAULT_MCP_COLLECTION_LIMIT, MAX_MCP_COLLECTION_LIMIT), + }; + const result = await findReferences( + snapshot.index, + { + file: await resolveProjectFile(await realRoot, root, file), + line, + column, + }, + referenceOptions, + ); + if (result.status !== "ok") return { references: [] }; + return { + references: result.references.map((reference) => ({ + file: relative(reference.file), + range: reference.range, + })), + }; + }), - deps: async (request) => { - const dependencies = await collectMcpDependencyEntries(request, getDependencies); - return { dependencies }; - }, + deps: async (request) => + await withFreshness(async () => { + const dependencies = await collectMcpDependencyEntries(request, getDependencies); + return { dependencies }; + }), - rdeps: async (request) => { - const reverseDependencies = await collectMcpDependencyEntries(request, getReverseDependencies); - return { reverseDependencies }; - }, + rdeps: async (request) => + await withFreshness(async () => { + const reverseDependencies = await collectMcpDependencyEntries(request, getReverseDependencies); + return { reverseDependencies }; + }), - path: async (request) => { - const snapshot = await session.loadProject({ symbolGraph: "skip" }); - const result = getShortestPath( - snapshot.fileGraph, - await resolveProjectFile(await realRoot, root, request.from), - await resolveProjectFile(await realRoot, root, request.to), - ); - return { - path: result ? result.map(relative) : null, - }; - }, + path: async (request) => + await withFreshness(async () => { + const snapshot = await session.loadProject({ symbolGraph: "skip" }); + const result = getShortestPath( + snapshot.fileGraph, + await resolveProjectFile(await realRoot, root, request.from), + await resolveProjectFile(await realRoot, root, request.to), + ); + return { + path: result ? result.map(relative) : null, + }; + }), impact: async (request) => - await buildReviewReport(root, { - ...options.buildOptions, - gitBase: request.base, - gitHead: request.head, - reviewDepth: "minimal", - }), + await withFreshness( + async () => + await buildReviewReport(root, { + ...options.buildOptions, + gitBase: request.base, + gitHead: request.head, + reviewDepth: "minimal", + }), + ), review: async (request) => - await buildReviewReport(root, { - ...options.buildOptions, - gitBase: request.base, - gitHead: request.head, - ...(request.reviewDepth !== undefined ? { reviewDepth: request.reviewDepth } : {}), - }), + await withFreshness( + async () => + await buildReviewReport(root, { + ...options.buildOptions, + gitBase: request.base, + gitHead: request.head, + ...(request.reviewDepth !== undefined ? { reviewDepth: request.reviewDepth } : {}), + }), + ), query_sqlite: async (request) => { if (!sqlitePath) { throw new Error("No SQLite artifact is available. Run artifact_build first or pass artifactPath."); } - const realSqlitePath = await assertRealPathCandidateWithinRoot(await realRoot, sqlitePath, "SQLite artifact"); assertMcpSqliteQueryResourceBounded(request.query); + // The artifact's own baseline decides whether the query can be served; session freshness + // only decides whether an automatic rebuild is safe. A fresh artifact is served even when + // the in-memory session snapshot is stale, and an unsafe session refresh burst blocks the + // automatic rebuild instead of silently rebuilding from stale in-memory state. + let realSqlitePath = await assertRealPathCandidateWithinRoot(await realRoot, sqlitePath, "SQLite artifact"); + let artifactFreshness = await checkSqliteArtifactFreshness(realSqlitePath); + if (artifactFreshness.state !== "fresh") { + const sessionFreshness = await checkMcpFreshness(); + if (sessionFreshness.state === "stale") { + throw new Error(formatSqliteFreshnessError(sessionFreshness)); + } + artifactFreshness = await refreshSqliteArtifactForQuery(artifactFreshness, { allowStaleRebuild: true }); + realSqlitePath = await assertRealPathCandidateWithinRoot(await realRoot, sqlitePath, "SQLite artifact"); + } const result = await queryGraphSqliteRaw(realSqlitePath, request.query, request.params ?? [], { maxRows: normalizeSqliteRowLimit(request.limit), }); - return boundRawSqlResult(result, DEFAULT_SQLITE_BYTE_LIMIT); + return { ...boundRawSqlResult(result, DEFAULT_SQLITE_BYTE_LIMIT), freshness: artifactFreshness }; }, refresh_index: async (request) => { const warmup = request.warmup ?? "off"; session.invalidate(); sqlitePath = configuredSqlitePath; + sqliteOutDir = configuredSqliteOutDir; + sqliteCanRefresh = configuredSqliteCanRefresh; await startCodegraphMcpWarmup(session, warmup); return { refreshed: true, warmup }; }, @@ -424,6 +655,16 @@ function createCodegraphMcpHandlersForSession( if (readOnly) { throw new Error("artifact_build is disabled in read-only MCP mode."); } + const freshness = await checkMcpFreshness(); + if (freshness.state === "stale") { + const changed = freshness.changedFiles.length ? ` Changed files: ${freshness.changedFiles.join(", ")}.` : ""; + const omitted = freshness.omittedChangedFileCount + ? ` Omitted changed files: ${freshness.omittedChangedFileCount}.` + : ""; + throw new Error( + `Cannot build artifacts from a stale MCP index; run refresh_index first. ${freshness.reason}.${changed}${omitted}`, + ); + } const outDir = request.outDir !== undefined ? await assertWritableDirectoryRealPathWithinRoot( @@ -446,8 +687,10 @@ function createCodegraphMcpHandlersForSession( const sqliteArtifact = result.artifacts.sqlite; if (sqliteArtifact) { sqlitePath = path.join(result.outDir, sqliteArtifact); + sqliteOutDir = result.outDir; + sqliteCanRefresh = true; } - return result; + return { ...result, freshness }; }, }; } diff --git a/src/sqlite.ts b/src/sqlite.ts index 49bb8b0b..ab8f0a23 100644 --- a/src/sqlite.ts +++ b/src/sqlite.ts @@ -1,3 +1,3 @@ export type { GraphQueryResult, RawSqlResult, SqliteGraphOptions, SqliteGraphUpdateOptions } from "./sqlite/types.js"; -export { writeGraphSqlite, updateGraphSqlite } from "./sqlite/write.js"; +export { SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY, writeGraphSqlite, updateGraphSqlite } from "./sqlite/write.js"; export { queryGraphSqlite, queryGraphSqliteRaw } from "./sqlite/query.js"; diff --git a/src/sqlite/types.ts b/src/sqlite/types.ts index ea8a5430..b360064e 100644 --- a/src/sqlite/types.ts +++ b/src/sqlite/types.ts @@ -5,6 +5,7 @@ export type SqliteGraphOptions = { fileGraph: Graph; symbolGraph: SymbolGraph; outputPath: string; + fileSignatures?: Iterable<{ path: string; size: number; mtimeMs: number }>; }; export type SqliteGraphUpdateOptions = { diff --git a/src/sqlite/write.ts b/src/sqlite/write.ts index 5162f59b..ea37c911 100644 --- a/src/sqlite/write.ts +++ b/src/sqlite/write.ts @@ -5,6 +5,37 @@ import type { SqliteGraphOptions, SqliteGraphUpdateOptions } from "./types.js"; import { execRowsParams } from "./common.js"; import { withSqliteDatabase } from "./database.js"; +export const SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY = "artifact_file_signatures_v1"; + +type SqliteArtifactFileSignature = { + path: string; + size: number; + mtimeMs: number; +}; + +function normalizeSqliteArtifactFileSignatures( + signatures: Iterable, +): SqliteArtifactFileSignature[] { + const normalized = [...signatures].map((signature) => ({ + path: signature.path, + size: signature.size, + mtimeMs: signature.mtimeMs, + })); + normalized.sort((left, right) => left.path.localeCompare(right.path)); + return normalized; +} + +function writeArtifactFileSignatures(db: SqliteDatabase, signatures: readonly SqliteArtifactFileSignature[]): void { + db.prepare("INSERT OR REPLACE INTO graph_metadata (key, value) VALUES (?, ?);").run([ + SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY, + JSON.stringify(signatures), + ]); +} + +function clearArtifactFileSignatures(db: SqliteDatabase): void { + db.prepare("DELETE FROM graph_metadata WHERE key = ?;").run([SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY]); +} + const collectSymbolIdsForFiles = (symbolGraph: SymbolGraph, changedSet: Set): Set => { const ids = new Set(); for (const [id, node] of symbolGraph.nodes.entries()) { @@ -212,6 +243,9 @@ const deleteUnreferencedExternalFiles = (db: SqliteDatabase) => { }; export async function writeGraphSqlite(options: SqliteGraphOptions): Promise { + const fileSignatures = options.fileSignatures + ? normalizeSqliteArtifactFileSignatures(options.fileSignatures) + : undefined; await withSqliteDatabase(options.outputPath, (db) => { const runInsert = db.transaction(() => { clearCurrentGraphState(db); @@ -239,6 +273,11 @@ export async function writeGraphSqlite(options: SqliteGraphOptions): Promise { expect(files).toContain(keptFile.replace(/\\/g, "/")); expect(files).not.toContain(ignoredFile.replace(/\\/g, "/")); }); + + it("reports stale file edits as normalized project display paths without invalidating cached project state", async () => { + const absoluteRoot = await fs.mkdtemp(path.join(os.tmpdir(), "cg-agent-session-check-fresh-")); + const relativeRoot = path.relative(process.cwd(), absoluteRoot); + const filePath = path.join(absoluteRoot, "src", "auth.ts"); + await fs.mkdir(path.dirname(filePath), { recursive: true }); + await fs.writeFile(filePath, "export function oldSymbol() { return 1; }\n", "utf8"); + const buildSpy = vi.spyOn(indexerBuild, "buildProjectIndexIncremental"); + const session = createAgentSession({ root: relativeRoot, useConfig: false, freshness: { policy: "check" } }); + const cached = await session.loadProject({ symbolGraph: "skip" }); + if (!session.checkFreshness) { + throw new Error("agent session should expose freshness checks"); + } + + await fs.writeFile(filePath, "export function editedSymbol() { return 22; }\n", "utf8"); + const freshness = await session.checkFreshness(); + const afterCheck = await session.loadProject({ symbolGraph: "skip" }); + + expect(freshness).toEqual({ + state: "stale", + changedFiles: ["src/auth.ts"], + changedFileCount: 1, + omittedChangedFileCount: 0, + reason: "session snapshot is older than files on disk", + }); + expect(afterCheck).toBe(cached); + expect(buildSpy).toHaveBeenCalledTimes(1); + }); + + it("counts deleted file bytes against auto-refresh limits", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-agent-session-delete-bytes-")); + const removedFile = path.join(root, "large.ts"); + await fs.writeFile(removedFile, `export const payload = "${"x".repeat(64)}";\n`, "utf8"); + const buildSpy = vi.spyOn(indexerBuild, "buildProjectIndexIncremental"); + const session = createAgentSession({ root, freshness: { policy: "auto", maxAutoRefreshBytes: 16 } }); + const cached = await session.loadProject({ symbolGraph: "skip" }); + if (!session.checkFreshness) { + throw new Error("agent session should expose freshness checks"); + } + + await fs.unlink(removedFile); + const freshness = await session.checkFreshness(); + const afterCheck = await session.loadProject({ symbolGraph: "skip" }); + + expect(freshness).toEqual({ + state: "stale", + changedFiles: ["large.ts"], + changedFileCount: 1, + omittedChangedFileCount: 0, + reason: "changed byte count exceeds 16", + }); + expect(afterCheck).toBe(cached); + expect(buildSpy).toHaveBeenCalledTimes(1); + }); }); diff --git a/tests/helpers/agent.ts b/tests/helpers/agent.ts index 90dbc799..1a3f4df7 100644 --- a/tests/helpers/agent.ts +++ b/tests/helpers/agent.ts @@ -4,25 +4,38 @@ export function countingSession(session: AgentSession): { session: AgentSession; let cached: Promise | undefined; let cachedMode: AgentLoadProjectOptions["symbolGraph"] | undefined; let loadCount = 0; - return { - session: { - ...(session.root ? { root: session.root } : {}), - ...(session.listFiles ? { listFiles: session.listFiles } : {}), - loadProject: async (options) => { - const mode = options?.symbolGraph ?? "eager"; - if (!cached || cachedMode !== mode) { - loadCount += 1; - cachedMode = mode; - cached = session.loadProject(options); - } - return await cached; - }, - invalidate: () => { + const countedSession: AgentSession = { + ...(session.root ? { root: session.root } : {}), + ...(session.listFiles ? { listFiles: session.listFiles } : {}), + ...(session.discoverFiles ? { discoverFiles: session.discoverFiles } : {}), + loadProject: async (options) => { + const mode = options?.symbolGraph ?? "eager"; + if (!cached || cachedMode !== mode) { + loadCount += 1; + cachedMode = mode; + cached = session.loadProject(options); + } + return await cached; + }, + invalidate: () => { + cached = undefined; + cachedMode = undefined; + session.invalidate(); + }, + }; + const checkFreshness = session.checkFreshness; + if (checkFreshness) { + countedSession.checkFreshness = async () => { + const freshness = await checkFreshness(); + if (freshness.state === "refreshed") { cached = undefined; cachedMode = undefined; - session.invalidate(); - }, - }, + } + return freshness; + }; + } + return { + session: countedSession, loads: () => loadCount, }; } diff --git a/tests/mcp-server.test.ts b/tests/mcp-server.test.ts index fa1c8484..0bb167e5 100644 --- a/tests/mcp-server.test.ts +++ b/tests/mcp-server.test.ts @@ -2,10 +2,17 @@ import fs from "node:fs/promises"; import { request as httpRequest } from "node:http"; import os from "node:os"; import path from "node:path"; +import { DatabaseSync } from "node:sqlite"; import { describe, expect, it, vi } from "vitest"; -import { createAgentSession } from "../src/agent/session.js"; -import { createCodegraphMcpHandlers, listCodegraphMcpTools, startCodegraphMcpHttpServer } from "../src/mcp/server.js"; +import { createAgentSession, type AgentSession } from "../src/agent/session.js"; +import { + createCodegraphMcpHandlers, + listCodegraphMcpTools, + startCodegraphMcpHttpServer, + type CodegraphMcpHandlers, +} from "../src/mcp/server.js"; import * as symbolGraphBuild from "../src/graphs/symbol-graph-detailed.js"; +import { SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY } from "../src/sqlite.js"; import { countingSession } from "./helpers/agent.js"; import { createArtifactOutputWithStaleFile, createLinkedTempRoot, isSymlinkUnavailable } from "./helpers/filesystem.js"; @@ -390,6 +397,258 @@ describe("codegraph MCP handlers", () => { expect(result.rowLimit).toBe(1); }); + it("refreshes the SQLite artifact before query_sqlite after small workspace edits", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-fresh-")); + await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); + const handlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await handlers.artifact_build({ outDir: path.join(root, "out"), sqlite: true }); + + await fs.writeFile(path.join(root, "two.ts"), "export const two = 2;\n"); + const result = await handlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + const paths = result.rows.map((row) => normalizeSqlitePath(row[0])); + + expect(paths.some((file) => file.endsWith("two.ts"))).toBe(true); + expect(result.freshness).toEqual({ state: "refreshed", changedFiles: ["two.ts"] }); + }); + + it("refreshes the SQLite artifact before query_sqlite after edits and deletions", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-fresh-edit-delete-")); + const keptFile = path.join(root, "one.ts"); + const removedFile = path.join(root, "remove.ts"); + await fs.writeFile(keptFile, "export function oldName() { return 1; }\n"); + await fs.writeFile(removedFile, "export function removedName() { return 2; }\n"); + const handlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await handlers.artifact_build({ outDir: path.join(root, "out"), sqlite: true }); + + await fs.writeFile(keptFile, "export function editedName() { return 3; }\n"); + await fs.unlink(removedFile); + const result = await handlers.query_sqlite({ query: "SELECT name FROM symbols ORDER BY name;" }); + const names = result.rows.map((row) => String(row[0])); + + expect(names).toContain("editedName"); + expect(names).not.toContain("oldName"); + expect(names).not.toContain("removedName"); + expect(result.freshness).toEqual({ state: "refreshed", changedFiles: ["one.ts", "remove.ts"] }); + }); + + it("guides stale SQLite rebuilds through refresh_index before artifact_build when the session is stale", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-stale-session-guidance-")); + const outDir = path.join(root, "out"); + const removedFile = path.join(root, "remove.ts"); + await fs.writeFile(path.join(root, "keep.ts"), "export const keep = 1;\n"); + await fs.writeFile(removedFile, `export const payload = "${"x".repeat(64)}";\n`); + const session = createAgentSession({ root, freshness: { policy: "auto", maxAutoRefreshBytes: 16 } }); + const handlers = createCodegraphMcpHandlers({ root, session, readOnly: false }); + await handlers.artifact_build({ outDir, sqlite: true }); + + await fs.unlink(removedFile); + let caught: unknown; + try { + await handlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + } catch (error) { + caught = error; + } + + expect(caught).toBeInstanceOf(Error); + if (!(caught instanceof Error)) { + throw new Error("query_sqlite should reject stale SQLite rebuilds with an Error"); + } + expect(caught.message).toMatch(/SQLite artifact is stale/); + expect(caught.message).toMatch(/refresh_index[\s\S]*artifact_build[\s\S]*query_sqlite/); + expect(caught.message).toMatch(/changed byte count exceeds 16/); + expect(caught.message).toMatch(/remove\.ts/); + expect(caught.message.indexOf("refresh_index")).toBeLessThan(caught.message.indexOf("artifact_build")); + expect(caught.message.indexOf("artifact_build")).toBeLessThan(caught.message.indexOf("query_sqlite")); + }); + + it("guides read-only stale SQLite artifacts to rebuild with write access enabled", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-stale-artifact-")); + const outDir = path.join(root, "out"); + await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); + const buildHandlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await buildHandlers.artifact_build({ outDir, sqlite: true }); + + await fs.writeFile(path.join(root, "two.ts"), "export const two = 2;\n"); + const readHandlers = createCodegraphMcpHandlers({ root, artifactPath: outDir, readOnly: true }); + let caught: unknown; + try { + await readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + } catch (error) { + caught = error; + } + + expect(caught).toBeInstanceOf(Error); + if (!(caught instanceof Error)) { + throw new Error("query_sqlite should reject stale SQLite artifacts with an Error"); + } + expect(caught.message).toMatch(/SQLite artifact is stale/); + expect(caught.message).toMatch(/rebuild[\s\S]*write access enabled/i); + expect(caught.message).toMatch(/two\.ts/); + expect(caught.message).not.toMatch(/artifact_build before query_sqlite/i); + }); + + it("guides read-only stale SQLite rebuilds through refresh_index when the session is stale", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-readonly-stale-session-")); + const outDir = path.join(root, "out"); + const removedFile = path.join(root, "remove.ts"); + await fs.writeFile(path.join(root, "keep.ts"), "export const keep = 1;\n"); + await fs.writeFile(removedFile, `export const payload = "${"x".repeat(64)}";\n`); + const session = createAgentSession({ root, freshness: { policy: "auto", maxAutoRefreshBytes: 16 } }); + const buildHandlers = createCodegraphMcpHandlers({ root, session, readOnly: false }); + await buildHandlers.artifact_build({ outDir, sqlite: true }); + + await fs.unlink(removedFile); + const readHandlers = createCodegraphMcpHandlers({ root, artifactPath: outDir, readOnly: true, session }); + let caught: unknown; + try { + await readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + } catch (error) { + caught = error; + } + + expect(caught).toBeInstanceOf(Error); + if (!(caught instanceof Error)) { + throw new Error("query_sqlite should reject stale read-only rebuilds with an Error"); + } + expect(caught.message).toMatch(/SQLite artifact is stale/); + expect(caught.message).toMatch(/refresh_index/i); + expect(caught.message).toMatch(/rebuild[\s\S]*write access enabled/i); + expect(caught.message).toMatch(/changed byte count exceeds 16/); + expect(caught.message).toMatch(/remove\.ts/); + const writeAccessIndex = caught.message.toLowerCase().indexOf("write access enabled"); + expect(caught.message.indexOf("refresh_index")).toBeLessThan(writeAccessIndex); + expect(caught.message).not.toMatch(/artifact_build before query_sqlite/i); + }); + + it("refuses older SQLite artifacts without freshness metadata in read-only mode", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-legacy-artifact-")); + const outDir = path.join(root, "out"); + await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); + const buildHandlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await buildHandlers.artifact_build({ outDir, sqlite: true }); + const db = new DatabaseSync(path.join(outDir, "codegraph.sqlite")); + try { + db.prepare("DELETE FROM graph_metadata WHERE key = ?;").run(SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY); + } finally { + db.close(); + } + const readHandlers = createCodegraphMcpHandlers({ root, artifactPath: outDir }); + + await expect(readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" })).rejects.toThrow( + /SQLite artifact has no freshness baseline/, + ); + }); + + it("rebuilds stale configured SQLite artifact bundles in writable mode", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-writable-bundle-")); + const outDir = path.join(root, "out"); + await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); + const buildHandlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await buildHandlers.artifact_build({ outDir, sqlite: true }); + const db = new DatabaseSync(path.join(outDir, "codegraph.sqlite")); + try { + db.prepare("DELETE FROM graph_metadata WHERE key = ?;").run(SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY); + } finally { + db.close(); + } + await fs.writeFile(path.join(root, "two.ts"), "export const two = 2;\n"); + const readHandlers = createCodegraphMcpHandlers({ root, artifactPath: outDir, readOnly: false }); + const result = await readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + const paths = result.rows.map((row) => normalizeSqlitePath(row[0])); + + expect(paths.some((file) => file.endsWith("two.ts"))).toBe(true); + expect(result.freshness).toEqual({ state: "refreshed", changedFiles: [] }); + }); + + it("refuses stale explicit SQLite artifact files in writable mode", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-explicit-file-")); + const outDir = path.join(root, "out"); + await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); + const buildHandlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await buildHandlers.artifact_build({ outDir, sqlite: true }); + + await fs.writeFile(path.join(root, "two.ts"), "export const two = 2;\n"); + const readHandlers = createCodegraphMcpHandlers({ + root, + artifactPath: path.join(outDir, "codegraph.sqlite"), + readOnly: false, + }); + + await expect(readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" })).rejects.toThrow( + /SQLite artifact is stale[\s\S]*two\.ts/, + ); + }); + + it("uses prebuilt session discovery for SQLite artifact freshness", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-session-discovery-")); + await fs.writeFile(path.join(root, "keep.ts"), "export const keep = 1;\n"); + await fs.writeFile(path.join(root, "ignored.ts"), "export const ignored = 2;\n"); + const session = createAgentSession({ + root, + discovery: { ignoreGlobs: ["ignored.ts"] }, + freshness: { policy: "auto" }, + }); + const handlers = createCodegraphMcpHandlers({ root, session, readOnly: false }); + await handlers.artifact_build({ outDir: path.join(root, "out"), sqlite: true }); + + const initial = await handlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + await fs.writeFile(path.join(root, "late.ts"), "export const late = 3;\n"); + const refreshed = await handlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + const initialPaths = initial.rows.map((row) => normalizeSqlitePath(row[0])); + const refreshedPaths = refreshed.rows.map((row) => normalizeSqlitePath(row[0])); + + expect(initialPaths.some((file) => file.endsWith("keep.ts"))).toBe(true); + expect(initialPaths.some((file) => file.endsWith("ignored.ts"))).toBe(false); + expect(initial.freshness).toEqual({ state: "fresh" }); + expect(refreshedPaths.some((file) => file.endsWith("late.ts"))).toBe(true); + expect(refreshedPaths.some((file) => file.endsWith("ignored.ts"))).toBe(false); + expect(refreshed.freshness).toEqual({ state: "refreshed", changedFiles: ["late.ts"] }); + }); + + it("rejects prebuilt SQLite sessions without live discovery", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-no-discovery-")); + const outDir = path.join(root, "out"); + await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); + const buildHandlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await buildHandlers.artifact_build({ outDir, sqlite: true }); + const backingSession = createAgentSession({ root }); + const session: AgentSession = { + loadProject: backingSession.loadProject, + checkFreshness: async () => ({ state: "fresh" }), + invalidate: backingSession.invalidate, + }; + const readHandlers = createCodegraphMcpHandlers({ root, artifactPath: outDir, session }); + + await expect(readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" })).rejects.toThrow( + /does not expose live file discovery/, + ); + }); + + it("serves a fresh SQLite artifact without checking session freshness", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-fresh-artifact-no-session-check-")); + await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); + const outDir = path.join(root, "out"); + const buildHandlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await buildHandlers.artifact_build({ outDir, sqlite: true }); + + const backingSession = createAgentSession({ root }); + let freshnessChecks = 0; + const session: AgentSession = { + ...backingSession, + checkFreshness: async () => { + freshnessChecks += 1; + throw new Error("query_sqlite should inspect fresh SQLite artifact metadata before session freshness"); + }, + }; + const readHandlers = createCodegraphMcpHandlers({ root, artifactPath: outDir, session }); + + const result = await readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + + expect(result.rows.map((row) => normalizeSqlitePath(row[0])).some((file) => file.endsWith("one.ts"))).toBe(true); + expect(result.freshness).toEqual({ state: "fresh" }); + expect(freshnessChecks).toBe(0); + }); + it("bounds query_sqlite bytes for MCP responses", async () => { const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-bytes-")); await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); @@ -441,19 +700,24 @@ describe("codegraph MCP handlers", () => { ).resolves.toEqual(expect.objectContaining({ rows: [[1]] })); }); - it("disables artifact builds by default and in explicit read-only mode", async () => { + it("disables artifact builds in read-only mode before checking freshness", async () => { const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-readonly-")); await fs.writeFile(path.join(root, "auth.ts"), "export function validateUser(id: number) { return id > 0; }\n"); + const backingSession = createAgentSession({ root }); + let freshnessChecks = 0; + const session: AgentSession = { + ...backingSession, + checkFreshness: async () => { + freshnessChecks += 1; + throw new Error("checkFreshness should not run before read-only artifact_build rejection"); + }, + }; - const defaultHandlers = createCodegraphMcpHandlers({ root }); - await expect(defaultHandlers.artifact_build({ outDir: path.join(root, "out"), sqlite: true })).rejects.toThrow( - /read-only/i, - ); - - const readOnlyHandlers = createCodegraphMcpHandlers({ root, readOnly: true }); + const readOnlyHandlers = createCodegraphMcpHandlers({ root, readOnly: true, session }); await expect(readOnlyHandlers.artifact_build({ outDir: path.join(root, "out"), sqlite: true })).rejects.toThrow( - /read-only/i, + /artifact_build is disabled in read-only MCP mode/i, ); + expect(freshnessChecks).toBe(0); }); it("rejects artifact paths outside the root", async () => { @@ -509,7 +773,7 @@ describe("codegraph MCP handlers", () => { const result = await handlers.get_file({ file: "large.txt", maxBytes: 5 }); - expect(result).toEqual({ + expect(result).toMatchObject({ file: "large.txt", text: "abcde", truncated: true, @@ -523,7 +787,7 @@ describe("codegraph MCP handlers", () => { const result = await handlers.get_file({ file: "unicode.txt", maxBytes: 5 }); - expect(result).toEqual({ + expect(result).toMatchObject({ file: "unicode.txt", text: "abc", truncated: true, @@ -662,7 +926,7 @@ describe("codegraph MCP handlers", () => { const scenarios: Array<{ name: string; - run: (handlers: ReturnType) => Promise; + run: (handlers: CodegraphMcpHandlers) => Promise; }> = [ { name: "goto", @@ -728,37 +992,112 @@ describe("codegraph MCP handlers", () => { expect(counted.loads()).toBe(0); }); - it("refresh_index invalidates stale MCP session snapshots", async () => { - const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-refresh-index-")); + it("auto-refreshes added files before MCP search", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-auto-fresh-add-")); await fs.writeFile(path.join(root, "auth.ts"), "export const ok = 1;\n", "utf8"); const handlers = createCodegraphMcpHandlers({ root }); const before = await handlers.search({ query: "lateSymbol", mode: "symbol", limit: 5 }); - await fs.writeFile(path.join(root, "late.ts"), "export const lateSymbol = 1;\n", "utf8"); - const stale = await handlers.search({ query: "lateSymbol", mode: "symbol", limit: 5 }); - const refresh = await handlers.refresh_index({ warmup: "base" }); - const fresh = await handlers.search({ query: "lateSymbol", mode: "symbol", limit: 5 }); + await fs.writeFile(path.join(root, "late.ts"), "export function lateSymbol() { return 1; }\n", "utf8"); + const after = await handlers.search({ query: "lateSymbol", mode: "symbol", limit: 5 }); expect(before.results.some((result) => result.label === "lateSymbol")).toBe(false); - expect(stale.results.some((result) => result.label === "lateSymbol")).toBe(false); - expect(refresh).toEqual({ refreshed: true, warmup: "base" }); - expect(fresh.results.some((result) => result.label === "lateSymbol")).toBe(true); + expect(after.results.some((result) => result.label === "lateSymbol")).toBe(true); + expect(after.freshness).toEqual({ state: "refreshed", changedFiles: ["late.ts"] }); }); - it("refresh_index reloads edited files in stale MCP session snapshots", async () => { - const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-refresh-edit-")); + it("auto-refreshes edited files before MCP search and reports root-relative changed paths", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-auto-fresh-edit-")); const filePath = path.join(root, "auth.ts"); - await fs.writeFile(filePath, "export const oldSymbol = 1;\n", "utf8"); + await fs.writeFile(filePath, "export function oldSymbol() { return 1; }\n", "utf8"); const handlers = createCodegraphMcpHandlers({ root }); - await handlers.search({ query: "oldSymbol", mode: "symbol", limit: 5 }); - await fs.writeFile(filePath, "export const editedSymbol = 1;\n", "utf8"); - const stale = await handlers.search({ query: "editedSymbol", mode: "symbol", limit: 5 }); - await handlers.refresh_index({}); - const fresh = await handlers.search({ query: "editedSymbol", mode: "symbol", limit: 5 }); + const before = await handlers.search({ query: "oldSymbol", mode: "symbol", limit: 5 }); + await fs.writeFile(filePath, "export function editedSymbol() { return 2; }\n", "utf8"); + const after = await handlers.search({ query: "editedSymbol", mode: "symbol", limit: 5 }); + + expect(before.results.some((result) => result.label === "oldSymbol")).toBe(true); + expect(after.results.some((result) => result.label === "editedSymbol")).toBe(true); + expect(after.freshness).toEqual({ state: "refreshed", changedFiles: ["auth.ts"] }); + }); + + it("auto-refreshes deleted files before MCP search", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-auto-fresh-delete-")); + const filePath = path.join(root, "remove.ts"); + await fs.writeFile(filePath, "export function removedSymbol() { return 1; }\n", "utf8"); + const handlers = createCodegraphMcpHandlers({ root }); + + const before = await handlers.search({ query: "removedSymbol", mode: "symbol", limit: 5 }); + await fs.unlink(filePath); + const after = await handlers.search({ query: "removedSymbol", mode: "symbol", limit: 5 }); + + expect(before.results.some((result) => result.label === "removedSymbol")).toBe(true); + expect(after.results.some((result) => result.label === "removedSymbol")).toBe(false); + expect(after.freshness).toEqual({ state: "refreshed", changedFiles: ["remove.ts"] }); + }); + + it("reports bounded stale metadata when MCP auto-refresh thresholds are exceeded", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-auto-fresh-burst-")); + await fs.writeFile(path.join(root, "auth.ts"), "export function cachedSymbol() { return 1; }\n", "utf8"); + const session = createAgentSession({ root, freshness: { policy: "auto", maxAutoRefreshFiles: 1 } }); + const handlers = createCodegraphMcpHandlers({ root, session }); + + await handlers.search({ query: "cachedSymbol", mode: "symbol", limit: 5 }); + await Promise.all( + Array.from({ length: 30 }, async (_, index) => { + const suffix = String(index).padStart(2, "0"); + await fs.writeFile(path.join(root, `late-${suffix}.ts`), `export const late${suffix} = ${index};\n`, "utf8"); + }), + ); + const after = await handlers.search({ query: "cachedSymbol", mode: "symbol", limit: 5 }); + + expect(after.results.some((result) => result.label === "cachedSymbol")).toBe(true); + expect(after.freshness).toEqual({ + state: "stale", + changedFiles: Array.from({ length: 25 }, (_, index) => `late-${String(index).padStart(2, "0")}.ts`), + changedFileCount: 30, + omittedChangedFileCount: 5, + reason: "changed file count exceeds 1", + }); + }); + + it("reports stale metadata when MCP auto-refresh byte thresholds are exceeded", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-auto-fresh-bytes-")); + const filePath = path.join(root, "auth.ts"); + await fs.writeFile(filePath, "export function cachedSymbol() { return 1; }\n", "utf8"); + const session = createAgentSession({ root, freshness: { policy: "auto", maxAutoRefreshBytes: 5 } }); + const handlers = createCodegraphMcpHandlers({ root, session }); + + await handlers.search({ query: "cachedSymbol", mode: "symbol", limit: 5 }); + await fs.writeFile(filePath, `export function largeSymbol() { return "${"x".repeat(64)}"; }\n`, "utf8"); + const after = await handlers.search({ query: "cachedSymbol", mode: "symbol", limit: 5 }); + + expect(after.results.some((result) => result.label === "cachedSymbol")).toBe(true); + expect(after.freshness).toEqual({ + state: "stale", + changedFiles: ["auth.ts"], + changedFileCount: 1, + omittedChangedFileCount: 0, + reason: "changed byte count exceeds 5", + }); + }); + + it("returns live file bytes and freshness metadata after MCP file edits", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-auto-fresh-read-")); + const filePath = path.join(root, "auth.ts"); + await fs.writeFile(filePath, "export function readBefore() { return 'old'; }\n", "utf8"); + const handlers = createCodegraphMcpHandlers({ root }); + + await handlers.search({ query: "readBefore", mode: "symbol", limit: 5 }); + await fs.writeFile(filePath, "export function readAfter() { return 'new bytes'; }\n", "utf8"); + const read = await handlers.get_file({ file: "auth.ts" }); - expect(stale.results.some((result) => result.label === "editedSymbol")).toBe(false); - expect(fresh.results.some((result) => result.label === "editedSymbol")).toBe(true); + expect(read).toEqual({ + file: "auth.ts", + text: "export function readAfter() { return 'new bytes'; }\n", + truncated: false, + freshness: { state: "fresh" }, + }); }); it("refresh_index clears stale SQLite artifact state", async () => { @@ -848,7 +1187,7 @@ describe("codegraph MCP handlers", () => { } }); - it("uses the MCP session snapshot when artifact_build is enabled", async () => { + it("refuses artifact_build when the MCP session snapshot is stale before creating output", async () => { const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-artifact-session-")); await fs.writeFile(path.join(root, "auth.ts"), "export function validateUser(id: number) { return id > 0; }\n"); const counted = countingSession(createAgentSession({ root })); @@ -856,12 +1195,12 @@ describe("codegraph MCP handlers", () => { await handlers.search({ query: "validate user", limit: 5 }); await fs.writeFile(path.join(root, "late.ts"), "export const late = 1;\n"); - await handlers.artifact_build({ outDir: path.join(root, "out"), graphJson: true }); + const outDir = path.join(root, "out"); - const graph = JSON.parse(await fs.readFile(path.join(root, "out", "graph.json"), "utf8")) as { - graph: { files: string[] }; - }; - expect(graph.graph.files.some((file) => file.endsWith("late.ts"))).toBe(false); + await expect(handlers.artifact_build({ outDir, graphJson: true })).rejects.toThrow( + /Cannot build artifacts from a stale MCP index[\s\S]*late\.ts/, + ); + await expect(fs.readdir(outDir)).rejects.toMatchObject({ code: "ENOENT" }); expect(counted.loads()).toBe(1); }); @@ -937,6 +1276,59 @@ describe("codegraph MCP handlers", () => { expect(paths.some((file) => file.endsWith("auth.ts"))).toBeTruthy(); expect(paths.some((file) => file.includes("/out/") || file.endsWith("/out/old.ts"))).toBe(false); }); + + it("query_sqlite treats corrupted SQLite freshness metadata as a missing baseline", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-sqlite-corrupt-metadata-")); + const outDir = path.join(root, "out"); + await fs.writeFile(path.join(root, "one.ts"), "export const one = 1;\n"); + const buildHandlers = createCodegraphMcpHandlers({ root, readOnly: false }); + await buildHandlers.artifact_build({ outDir, sqlite: true }); + + const db = new DatabaseSync(path.join(outDir, "codegraph.sqlite")); + try { + const updated = db + .prepare("UPDATE graph_metadata SET value = ? WHERE key = ?;") + .run("this-is-not-json", SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY); + if (Number(updated.changes) === 0) { + db.prepare("INSERT OR REPLACE INTO graph_metadata (key, value) VALUES (?, ?);").run( + SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY, + "this-is-not-json", + ); + } + } finally { + db.close(); + } + + const readHandlers = createCodegraphMcpHandlers({ root, artifactPath: outDir }); + + await expect(readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" })).rejects.toThrow( + /no freshness baseline/i, + ); + + let caught: unknown; + try { + await readHandlers.query_sqlite({ query: "SELECT path FROM files ORDER BY path;" }); + } catch (error) { + caught = error; + } + expect(caught).toBeInstanceOf(Error); + expect((caught as Error).message).not.toMatch(/JSON|Unexpected token/i); + }); + + it("reports fresh when a prebuilt MCP session has no freshness check", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "cg-mcp-session-no-freshness-check-")); + await fs.writeFile(path.join(root, "one.ts"), "export function lonelySymbol() { return 1; }\n"); + const backingSession = createAgentSession({ root }); + const session: AgentSession = { + loadProject: backingSession.loadProject, + invalidate: backingSession.invalidate, + }; + const handlers = createCodegraphMcpHandlers({ root, session }); + + const result = await handlers.search({ query: "lonelySymbol", mode: "symbol", limit: 5 }); + + expect(result.freshness).toEqual({ state: "fresh" }); + }); }); function normalizeSqlitePath(value: unknown): string { diff --git a/tests/sqlite.test.ts b/tests/sqlite.test.ts index 1c698d95..f82106c8 100644 --- a/tests/sqlite.test.ts +++ b/tests/sqlite.test.ts @@ -10,6 +10,7 @@ import { updateGraphSqlite, queryGraphSqlite, queryGraphSqliteRaw, + SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY, } from "../src/index.js"; import { mkTmpDir, normalizeTestPath } from "./helpers/filesystem.js"; @@ -20,6 +21,13 @@ const dbQuery = (db: DatabaseSync, sql: string): string[] => { return rows.map((row) => String(row[0])); }; +const artifactFreshnessMetadataRows = async (dbPath: string): Promise>> => { + const result = await queryGraphSqliteRaw(dbPath, "SELECT value FROM graph_metadata WHERE key = ?;", [ + SQLITE_ARTIFACT_FILE_SIGNATURES_METADATA_KEY, + ]); + return result.rows; +}; + describe("SQLite graph export", () => { it("writes tables, indexes, and supports basic queries", async () => { const root = await mkTmpDir("dg-sqlite-"); @@ -70,6 +78,57 @@ export function run() { helper(); new Widget(); } db.close(); }); + it("clears artifact freshness metadata on unsigned full rewrites", async () => { + const root = await mkTmpDir("dg-sqlite-freshness-rewrite-"); + const mainPath = path.join(root, "main.ts"); + await fsp.writeFile(mainPath, "export const one = 1;\n", "utf8"); + const index = await buildProjectIndex(root); + const sgraph = await buildSymbolGraphDetailed(index); + const dbPath = path.join(root, "graph.sqlite"); + const stat = await fsp.stat(mainPath); + await writeGraphSqlite({ + fileGraph: index.graph, + symbolGraph: sgraph, + outputPath: dbPath, + fileSignatures: [{ path: mainPath, size: stat.size, mtimeMs: stat.mtimeMs }], + }); + expect(await artifactFreshnessMetadataRows(dbPath)).toHaveLength(1); + + await writeGraphSqlite({ + fileGraph: index.graph, + symbolGraph: sgraph, + outputPath: dbPath, + }); + expect(await artifactFreshnessMetadataRows(dbPath)).toHaveLength(0); + }); + + it("clears artifact freshness metadata on incremental updates", async () => { + const root = await mkTmpDir("dg-sqlite-freshness-update-"); + const mainPath = path.join(root, "main.ts"); + await fsp.writeFile(mainPath, "export const one = 1;\n", "utf8"); + let index = await buildProjectIndex(root); + let sgraph = await buildSymbolGraphDetailed(index); + const dbPath = path.join(root, "graph.sqlite"); + const stat = await fsp.stat(mainPath); + await writeGraphSqlite({ + fileGraph: index.graph, + symbolGraph: sgraph, + outputPath: dbPath, + fileSignatures: [{ path: mainPath, size: stat.size, mtimeMs: stat.mtimeMs }], + }); + expect(await artifactFreshnessMetadataRows(dbPath)).toHaveLength(1); + await fsp.writeFile(mainPath, "export const two = 2;\n", "utf8"); + index = await buildProjectIndex(root); + sgraph = await buildSymbolGraphDetailed(index); + await updateGraphSqlite({ + fileGraph: index.graph, + symbolGraph: sgraph, + outputPath: dbPath, + changedFiles: [normalizeTestPath(mainPath)], + }); + expect(await artifactFreshnessMetadataRows(dbPath)).toHaveLength(0); + }); + it("keeps full SQLite exports idempotent across repeated writes", async () => { const root = await mkTmpDir("dg-sqlite-idempotent-"); await fsp.writeFile(