Skip to content

feat: tiered show/snippet lookup fast paths #184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SutuSebastian merged 7 commits into from
Jun 11, 2026
Merged

feat: tiered show/snippet lookup fast paths #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
0719837
feat: tiered show lookup fast paths and covering index
SutuSebastian Jun 11, 2026
b77c987
harden: lift tiered lookup docs across consumer surfaces
SutuSebastian Jun 11, 2026
1c3d4c5
harden: snippet help and rule parity for lookup tiers
SutuSebastian Jun 11, 2026
f0a317d
harden: consumer-surface parity and stricter EXPLAIN test
SutuSebastian Jun 11, 2026
17f0f9b
fix: align show/snippet integration tests with lookup tiers
SutuSebastian Jun 11, 2026
adae1c8
chore: run paired unit tests when staging source files
SutuSebastian Jun 11, 2026
b97193a
harden: lint-staged test globs and verify-after-each-step parity
SutuSebastian Jun 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions .agents/rules/verify-after-each-step.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,14 @@ AI agents tend to chain many edits across files and only discover breakage at co

## Current Per-File Checks (from `lint-staged.config.js`)

| File pattern | Checks |
| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `*.{js,jsx,ts,tsx,mjs,mts,cjs,cts}` | `bun run format:check`, `bun run lint` |
| `*.{css,json,md,mdc,html,yaml,yml}` | `bun run format:check` |
| `*.{ts,tsx}` | `bun run typecheck` with a temporary `tsconfig.lint-staged.json` that includes only **staged files under `src/`** (project-wide types still interconnect — use `bun run typecheck` if you need full-project certainty) |
| `*.test.ts` | `bun test` (on changed test files) |
| File pattern | Checks |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `*.{js,jsx,ts,tsx,mjs,mts,cjs,cts}` | `bun run format:check`, `bun run lint` |
| `*.{css,json,md,mdc,html,yaml,yml}` | `bun run format:check` |
| `*.{ts,tsx}` | `bun run typecheck` with a temporary `tsconfig.lint-staged.json` that includes only **staged files under `src/`** (project-wide types still interconnect — use `bun run typecheck` if you need full-project certainty); **`bun test`** on co-located `*.test.{ts,tsx}` pairs when a staged `src/` source file is present but its test file is not |
| `*.test.ts` | `bun test` (on changed test files) |
| `*.test.tsx` | `bun test` (on changed test files) |
| `scripts/**/*.test.mjs` | `bun test` (on changed test files) |

## What Counts as a Step

Expand All @@ -41,6 +43,6 @@ A "step" is any self-contained unit of work where you've finished editing and ar
1. **Verify after every step** — Run the matching checks on every file you touched during that step before moving to the next one.
2. **Fix before moving on** — If any check fails, fix it immediately while context is fresh. Never carry forward known failures.
3. **Use the right scope** — Run `bun run lint` and `bun run format:check` on specific files when possible. Prefer `bun run typecheck` project-wide when types may depend on unstaged files.
4. **Run affected tests** — If you modified or created `*.test.ts` files, run `bun test <file>` on them.
4. **Run affected tests** — If you modified or created `*.test.ts` / `*.test.tsx` / `scripts/**/*.test.mjs` files, run `bun test <file>` on them; if you changed a `src/` source file with a co-located test, run that pair even when the test file itself is unstaged.
5. **Re-index before querying Codemap** — If you changed indexed source and plan to run SQL against the structural index next, run `bun src/index.ts --files <paths>` with paths **relative to the indexed project root** (set `CODEMAP_TEST_BENCH` / `CODEMAP_ROOT` or `--root` so that root is correct — see [docs/benchmark.md § Indexing another project](../../docs/benchmark.md#indexing-another-project)).
6. **Don't duplicate the hook's job** — You don't need to re-verify at commit time; the pre-commit hook (`lint-staged`) handles that automatically when AI/agent env vars trigger it. Your job is to stay green _between_ commits.
5 changes: 5 additions & 0 deletions .changeset/tiered-lookup-fast-paths.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@stainless-code/codemap": patch
---

`show` and `snippet` now use fast equality lookup for exact `name` and lone `name:Token` queries (no wildcards); substring, multi-field, and FTS paths stay on the broader slow tier. CLI help, MCP tool descriptions, and bundled agent guidance document the two tiers.
29 changes: 15 additions & 14 deletions docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ Three **mutually exclusive** CLI entry shapes; all converge on `applyDiffPayload

**Backlog (not rejected):** `organize-imports` diff-shape recipe; `codemap-to-tsmorph` Path B adapter — tracked in [roadmap.md § Backlog](./roadmap.md#backlog). **Tracked elsewhere:** C.9 entry-point integration — [`plans/c9-plugin-layer.md`](./plans/c9-plugin-layer.md).

**Show / snippet wiring:** **`src/cli/show-snippet-args.ts`** (shared argv parser) + **`src/cli/show-snippet-render.ts`** (shared terminal/JSON error helpers) + **`src/cli/cmd-show.ts`** + **`src/cli/cmd-snippet.ts`** — sibling CLI verbs sharing the same parser shape (`<name>` or **`--query '<field:value …>'`** + **`--with-fts`** + `--kind` + `--in <path>` + `--json`; show adds **`--print-sql`**) and the pure engines **`src/application/show-engine.ts`** (exact lookup + envelope builders), **`src/application/search-query-parser.ts`** + **`src/application/search-engine.ts`** (field-qualified search → parameterized SQL on `symbols`, optional `source_fts` join), and **`src/application/show-search-mode.ts`** (shared parse/normalize + FTS resolution + **`executeShowLookup`** + **`formatShowSearchSqlForQuery`** for CLI/MCP/HTTP). Exact lookup: `findSymbolsByName({db, name, kind?, inPath?})`. Query lookup: `searchSymbols({db, parsed, withFts?})`. Snippet FS read: `readSymbolSource({match, projectRoot, indexedContentHash?})` + `getIndexedContentHash(db, filePath)`. **`buildShowResult`** + **`buildSnippetResult`** envelope builders — same engines the MCP show/snippet tools call. Both verbs return the same `{matches, disambiguation?, warning?}` envelope — single match → `{matches: [{...}]}`; multi-match adds `{n, by_kind, files, hint}`; optional **`warning`** when FTS was requested but `source_fts` is empty. Snippet matches add `source` / `stale` / `missing` fields (additive — no shape divergence). **`--in <path>`** and **`path:`** inside **`--query`** normalize through `toProjectRelative(projectRoot, p)` (from **`src/application/validate-engine.ts`**). Stale-file behavior on `snippet`: `hashContent` (from **`src/hash.ts`**) compares on-disk content against `files.content_hash`; mismatch sets `stale: true` but source IS still returned. MCP tools `show` and `snippet` register parallel to the CLI surface (see [§ MCP wiring](#cli-usage)).
**Show / snippet wiring:** **`src/cli/show-snippet-args.ts`** (shared argv parser) + **`src/cli/show-snippet-render.ts`** (shared terminal/JSON error helpers) + **`src/cli/cmd-show.ts`** + **`src/cli/cmd-snippet.ts`** — sibling CLI verbs sharing the same parser shape (`<name>` or **`--query '<field:value …>'`** + **`--with-fts`** + `--kind` + `--in <path>` + `--json`; show adds **`--print-sql`**) and the pure engines **`src/application/show-engine.ts`** (exact lookup + envelope builders), **`src/application/search-query-parser.ts`** + **`src/application/search-engine.ts`** (field-qualified search → parameterized SQL on `symbols`, optional `source_fts` join), and **`src/application/show-search-mode.ts`** (shared parse/normalize + FTS resolution + tiered routing via **`resolveExactNameFromParsedQuery`** / **`isExactNamePattern`** + **`executeShowLookup`** + **`formatShowSearchSqlForQuery`** for CLI/MCP/HTTP). **Fast tier:** positional `<name>` or lone `name:Token` without `%`/`_` wildcards (no `kind`/`path`/`in`/free text) → `findSymbolsByName` (`name = ?`, **`idx_symbols_name_covering`**). **Slow tier:** `name LIKE` substring, multi-field query, or FTS free-text → `searchSymbols`. Exact lookup with filters: `findSymbolsByName({db, name, kind?, inPath?})`. Snippet FS read: `readSymbolSource({match, projectRoot, indexedContentHash?})` + `getIndexedContentHash(db, filePath)`. **`buildShowResult`** + **`buildSnippetResult`** envelope builders — same engines the MCP show/snippet tools call. Both verbs return the same `{matches, disambiguation?, warning?}` envelope — single match → `{matches: [{...}]}`; multi-match adds `{n, by_kind, files, hint}`; optional **`warning`** when FTS was requested but `source_fts` is empty. Snippet matches add `source` / `stale` / `missing` fields (additive — no shape divergence). **`--in <path>`** and **`path:`** inside **`--query`** normalize through `toProjectRelative(projectRoot, p)` (from **`src/application/validate-engine.ts`**). Stale-file behavior on `snippet`: `hashContent` (from **`src/hash.ts`**) compares on-disk content against `files.content_hash`; mismatch sets `stale: true` but source IS still returned. MCP tools `show` and `snippet` register parallel to the CLI surface (see [§ MCP wiring](#cli-usage)).

**Evidence columns (high-judgment recipes):** Some bundled recipes add optional **`reason`** and **`evidence_json`** TEXT columns on each result row — factual detection path for agents, not pass/fail verdicts. Contract: [golden-queries.md § Evidence columns](./golden-queries.md#evidence-columns-high-judgment-recipes).

Expand Down Expand Up @@ -1053,16 +1053,17 @@ A covering index includes all columns needed by a query, so SQLite never touches

Key covering indexes:

| Index | Columns | Covers |
| ------------------------ | --------------------------------------------------------------------- | -------------------------- |
| `idx_symbols_name` | `name, kind, file_path, line_start, line_end, signature, is_exported` | Symbol lookup by name |
| `idx_imports_source` | `source, file_path` | "Who imports X?" queries |
| `idx_imports_resolved` | `resolved_path, file_path` | Resolved path lookups |
| `idx_exports_name` | `name, file_path, kind, is_default` | Export lookup by name |
| `idx_components_name` | `name, file_path, props_type, hooks_used` | Component search by name |
| `idx_components_file` | `file_path, name` | Components in a directory |
| `idx_dependencies_to` | `to_path, from_path` | Reverse dependency lookups |
| `idx_markers_kind` | `kind, file_path, line_number, content` | Marker listing by kind |
| `idx_css_variables_name` | `name, value, scope, file_path` | CSS token lookup by name |
| `idx_css_classes_name` | `name, file_path, is_module` | CSS class lookup |
| `idx_css_keyframes_name` | `name, file_path` | Keyframe lookup |
| Index | Columns | Covers |
| --------------------------- | ---------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| `idx_symbols_name` | `name, kind, file_path, line_start, line_end, signature, is_exported` | Symbol lookup by name |
| `idx_symbols_name_covering` | `name, kind, file_path, line_start, line_end, signature, is_exported, parent_name, visibility` | Full `findSymbolsByName` SELECT without table lookup |
| `idx_imports_source` | `source, file_path` | "Who imports X?" queries |
| `idx_imports_resolved` | `resolved_path, file_path` | Resolved path lookups |
| `idx_exports_name` | `name, file_path, kind, is_default` | Export lookup by name |
| `idx_components_name` | `name, file_path, props_type, hooks_used` | Component search by name |
| `idx_components_file` | `file_path, name` | Components in a directory |
| `idx_dependencies_to` | `to_path, from_path` | Reverse dependency lookups |
| `idx_markers_kind` | `kind, file_path, line_number, content` | Marker listing by kind |
| `idx_css_variables_name` | `name, value, scope, file_path` | CSS token lookup by name |
| `idx_css_classes_name` | `name, file_path, is_module` | CSS class lookup |
| `idx_css_keyframes_name` | `name, file_path` | Keyframe lookup |
6 changes: 3 additions & 3 deletions docs/glossary.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ TS shape for the JSON emitted by `codemap context`. Stable contract; agents can

### covering index

A SQLite index that includes every column needed by a query, so SQLite reads everything from the index B-tree without touching the main table. The query plan shows `USING COVERING INDEX`. Used heavily for AI agent query patterns — see [architecture § Covering indexes](./architecture.md#covering-indexes).
A SQLite index that includes every column needed by a query, so SQLite reads everything from the index B-tree without touching the main table. The query plan shows `USING COVERING INDEX`. Used heavily for AI agent query patterns — e.g. `idx_symbols_name_covering` for `codemap show` equality lookups. See [architecture § Covering indexes](./architecture.md#covering-indexes).

### `css_classes` (table)

Expand Down Expand Up @@ -543,11 +543,11 @@ Long-running transport shutdown rules in `src/application/session-lifecycle.ts`.

### show

`codemap show <name>` — one-step lookup that returns metadata (`file_path:line_start-line_end` + `signature` + `kind`) for symbol(s). **Exact mode:** `<name>` is case-sensitive; flags `--kind`, `--in`. **Field-qualified mode:** `--query 'kind:… name:… path:… in:…'` with optional free text; `--with-fts` (or `fts5: true` when indexed) searches file bodies via `source_fts` and returns every symbol in matching files; `--print-sql` prints Moat-A equivalent SQL. Output: `{matches, disambiguation?, warning?}` (`warning` when FTS was requested but `source_fts` is empty). MCP: `show` with `{name}` or `{query, with_fts?}`. Distinct from **snippet** (adds source text) and from hand-composed `query` SQL. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).
`codemap show <name>` — one-step lookup that returns metadata (`file_path:line_start-line_end` + `signature` + `kind`) for symbol(s). **Fast tier (equality index):** positional `<name>` or lone `name:Token` with no `%`/`_` wildcards and no other query fields — same rows as exact `<name>`. **Slow tier:** `name:%pat%` substring LIKE, multi-field `--query`, or free text (`name LIKE` / `source_fts` with `--with-fts` or `fts5: true`). **Exact mode flags:** `--kind`, `--in` (positional only). `--print-sql` prints Moat-A equivalent SQL. Output: `{matches, disambiguation?, warning?}` (`warning` when FTS was requested but `source_fts` is empty). MCP: `show` with `{name}` or `{query, with_fts?}`. Distinct from **snippet** (adds source text) and from hand-composed `query` SQL. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).

### snippet

`codemap snippet <name>` — same lookup modes as **show** (`<name>` + `--kind` / `--in`, or `--query` + `--with-fts`), but each match also carries `source` (file lines from disk at `line_start..line_end`), `stale` (true when content_hash drifted since last index), and `missing` (true when file is gone). Envelope: `{matches, disambiguation?, warning?}` with the same additive fields on each row. Stale-file behavior: `source` is always returned when the file exists; `stale: true` is metadata the agent reads (no auto-reindex). MCP: `snippet` with `{name}` or `{query, with_fts?}`. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).
`codemap snippet <name>` — same fast/slow lookup tiers as **show** (`<name>` + `--kind` / `--in`, or `--query` + `--with-fts`), but each match also carries `source` (file lines from disk at `line_start..line_end`), `stale` (true when content_hash drifted since last index), and `missing` (true when file is gone). Envelope: `{matches, disambiguation?, warning?}` with the same additive fields on each row. Stale-file behavior: `source` is always returned when the file exists; `stale: true` is metadata the agent reads (no auto-reindex). MCP: `snippet` with `{name}` or `{query, with_fts?}`. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).

### `suppressions` (table) / `// codemap-ignore-next-line` / `// codemap-ignore-file`

Expand Down
Loading
Loading