Multi-language, trait-based code-analysis metamodel (FamixNG-style), implemented in TypeScript. Language-specific extractors (Java first, via Spoon) emit a JSON interchange file; a TypeScript analyzer validates it, builds the dependency graph, and runs analyses (imports, coupling, cycles, architecture).
Source of truth for design decisions: the August 2026 exploration document (traits, not hierarchy; provenance on every edge; imports as the first-class comparable layer; containment ≠ attachment; stubs for external types).
┌─────────────────────┐ ┌──────────────────────┐ ┌─────────────────────┐
│ Extractors │ │ JSON interchange │ │ TypeScript analyzer │
│ (per language, │ ──▶ │ contract │ ──▶ │ validate → graph → │
│ native tooling) │ │ model.json │ │ queries → exports │
│ │ │ (versioned schema) │ │ │
│ Java: Spoon (JVM) │ └──────────────────────┘ └─────────────────────┘
│ Clojure: clj-kondo │ ▲
│ TS/JS: ts compiler │ │ JSON Schema published from the
│ … (later) │ │ TS core package → extractors in any
└─────────────────────┘ │ language can self-validate output
Key principle: extractors are federated behind one contract. Each extractor uses the best native tool for its language and only has to produce conforming JSON. All intelligence about the metamodel (traits, profiles, validation, derived indexes, analyses) lives once, in TypeScript.
codegraph/
├── PLAN.md
├── package.json # pnpm workspace root
├── pnpm-workspace.yaml
├── tsconfig.base.json
├── packages/
│ ├── core/ # @codegraph/core — metamodel: traits, kinds,
│ │ # entities, edges, profiles, validation,
│ │ # JSON Schema export
│ ├── analyzer/ # @codegraph/analyzer — graph construction,
│ │ # derived indexes, queries, metrics, exports
│ ├── cli/ # @codegraph/cli — `codegraph` command
│ └── viz/ # (future) Three.js "code city" 3D visualization
│ # of the model — the only package allowed
│ # to depend on three
├── extractors/
│ └── java/ # Maven/Gradle project, Spoon-based, emits model.json
│ └── src/main/java/...
├── schemas/ # generated JSON Schema files (committed,
│ # versioned — the cross-language contract)
└── fixtures/
└── java/ # small reference corpus + expected model.json
Tooling: Node ≥ 22, pnpm, TypeScript strict, Zod v4 (runtime validation + static types + native JSON Schema export), Vitest + fast-check (property-based tests), tsup for builds.
Why Zod: one definition per trait yields (a) the inferred TS type, (b) the
runtime validator, (c) z.toJSONSchema() output for non-TS extractors — the
exact role Malli plays in the original Clojure design.
- pnpm workspace,
tsconfig.base.json(strict,"module": "NodeNext"), ESLint, Vitest. - Empty
core,analyzer,clipackages wired together; CI script (pnpm -r build && pnpm -r test). - Rewrite README.md (currently GitLab template) with project pitch + this plan's summary.
// Opaque entity id: "<lang>:<module>/<symbol>[#<disambiguator>]"
// disambiguator = signature (overloads, receivers) or "file:startLine" for anonymous entities.
type EntityId = string; // opaque — never parsed by the analyzer, only compared
const SourceAnchor = z.object({
file: z.string(),
span: z.tuple([z.number().int(), z.number().int()]), // [startLine, endLine]
});
const Provenance = z.enum(["declared", "derived", "dynamic-candidate", "generated"]);Each trait declares only the keys it contributes:
| Trait | Keys contributed |
|---|---|
TNamed |
name |
TSourceAnchor |
anchor: SourceAnchor |
TComment |
comments: string[] |
TWithChildren |
children: EntityId[] (lexical containment) |
TChildOf |
parent: EntityId |
TAttachedTo |
attachedTo: EntityId (semantic attachment — Go receivers, Rust impl blocks, C# extension methods, Clojure extend-type) |
TModule |
definedIn: string[] (CodeFile paths; 1-1, 1-N or N-N per language), isStub: boolean |
TType |
isStub: boolean |
TWithInheritances |
(marker — edges carry the data) |
TWithImplements |
(marker — edges carry the data) |
TTypedEntity |
declaredType?: EntityId |
TInvocable |
signature: string |
TWithParameters |
parameters: EntityId[] |
TWithLocalVariables |
localVariables: EntityId[] |
TWithInvocations |
(marker — outgoing Invocation edges) |
TStructural |
(marker — value holder) |
TWithAccesses |
(marker — outgoing Access edges) |
Design rules carried over verbatim:
- Outgoing only — incoming indexes are always derived by the analyzer, never serialized (consistency + no serialization cycles).
- Containment ≠ attachment —
TChildOf/TWithChildren(where it's written) vsTAttachedTo(what it semantically belongs to) are distinct. space?: ("type" | "value")[]on entities for TypeScript profiles (interface = type-space only; class = both) — optional field, only meaningful in TS/… profiles.
Implementation: TRAITS: Record<TraitName, ZodObject> + a TraitName string
literal union. An Entity is:
const EntityBase = z.object({
id: z.string(),
kind: z.string(), // constrained per profile, not globally
traits: z.array(TraitName),
}).passthrough(); // trait keys validated per declared traitAll edges carry from, to, provenance, anchor (evidence), optional
sourceFile (C# partial classes), optional candidates: EntityId[]
(uncertain dispatch).
Edge kinds: import, inheritance, interfaceImplementation, invocation,
access (+ isRead/isWrite), reference, embedding (Go), traitUsage
(PHP), fileInclude (PHP). Discriminated union on edge field.
interface Profile {
lang: string;
kinds: Record<string, { required: TraitName[]; optional: TraitName[] }>;
edges: EdgeKind[]; // which associations this language can emit
space?: Record<string, Space[]>; // per kind, the declaration spaces it MAY occupy
notes?: string[]; // documented static-analysis blind spots
}Decision (M1 audit, added after the first draft of this section): space?
belongs on the Profile, not only on the Entity. METAMODEL §1.4 says the type/value
split is "only meaningful in profiles that declare it"; without a profile-side
declaration that sentence is unenforceable and space: ["type"] on a Java entity
would validate. A profile that omits space licenses none, which is exactly the
statement "this language has no type/value split". Only the TypeScript profile
declares it. The field is optional and additive: every profile literal predating
the decision still compiles unchanged.
Decision (was open in the design doc): validation uses
required ⊆ traits ⊆ required ∪ optional — strict equality is too brittle for
optional traits like TComment/TSourceAnchor; unrestricted subset would hide
extractor bugs. This gives both.
- Define all 9 profiles (Java, C#, Go, Clojure, JS, TS, Python, PHP, Rust) as data files — specifiable without being implemented (robustness test). Only Java's needs to be exercised in Phase 2.
Decision (M1 review) — lang ids are frozen as declared:
clj |
csharp |
go |
java |
js |
php |
python |
rust |
ts |
|---|
Abbreviated where the abbreviation is the idiomatic name of the language
(clj, js, ts), spelled out otherwise. The mixed style is deliberate and
not to be "tidied": the lang id is the EntityId prefix, so it is embedded in
every id an extractor emits and in every edge endpoint referencing one. Renaming
one after an extractor ships invalidates that extractor's whole id space and any
model.json already produced. Frozen before M2 for exactly that reason.
validateEntity(profile, entity):
1. entity.kind ∈ profile.kinds
2. required(kind) ⊆ entity.traits ⊆ required(kind) ∪ optional(kind)
3. ∀ t ∈ entity.traits: TRAITS[t].parse(entity) succeeds
validateModel(model):
+ every edge's provenance is set
+ edge kinds allowed by the profile
(graph closure — every from/to/parent/children id resolves — is checked in
the analyzer, since stubs are legitimate targets)Reference example (the doc's §9 EDN example, translated):
{
"id": "java:com.acme.order/OrderService.bill(com.acme.order.Order)",
"kind": "method",
"traits": ["TNamed", "TInvocable", "TWithParameters", "TWithLocalVariables",
"TWithInvocations", "TWithAccesses", "TTypedEntity", "TChildOf",
"TSourceAnchor"],
"name": "bill",
"signature": "bill(com.acme.order.Order)",
"declaredType": "java:com.acme.billing/Invoice",
"parent": "java:com.acme.order/OrderService",
"anchor": { "file": "OrderService.java", "span": [15, 22] }
}{
"edge": "invocation",
"from": "java:com.acme.order/OrderService.bill(com.acme.order.Order)",
"to": "java:com.acme.order/TaxCalculator.apply(double)",
"candidates": ["java:com.acme.order/TaxCalculator.apply(double)"],
"provenance": "declared",
"anchor": { "file": "OrderService.java", "span": [19, 19] }
}Id parameter types are erased FQNs, not simple names (M2, verified against
Spoon). archive(java.util.List) and archive(com.acme.order.legacy.List) are
legal overloads whose simple names are identical; rendered as archive(List)
they collapse into one id and one method disappears silently. Ids must be
unique per model, so the FQN form is normative — here, in METAMODEL.md §10, and
in the extractor. The fixture corpus pins the collision as a regression test.
-
pnpm run gen:schemas→z.toJSONSchema()→schemas/model.schema.json(committed; the Java extractor validates against it in its own tests).
Decision (M1 audit): the published schema must be sufficient on its own. Zod
refinements do not survive z.toJSONSchema(), so the trait-key rule of §4.5 step 3
is re-stated in the emitted schema as one if/then conditional per key-contributing
trait, generated from the same TRAITS table. Without it the contract accepted
{traits: ["TNamed"]} with no name, and "an extractor in any language can
self-validate using only the published schema" was only partly true. What remains
outside the schema — deliberately — is everything profile-aware: which kinds exist
and which trait compositions each kind licenses. That is validateEntity's job and
requires the profile data, so an extractor self-validates structure + vocabulary +
trait keys against the schema, and the analyzer adds the profile pass on load.
Deliverable Phase 1: @codegraph/core published locally; 9 profile data
files; JSON Schema generated; unit tests incl. the canonical hierarchy-breaking
case (a Clojure fn-var validating as TNamed + TStructural + TInvocable).
Separate Maven project in extractors/java/, JDK 17+, Spoon
in noClasspath mode (legacy, non-compilable corpora). Output: model.json
via Jackson. CLI: java -jar codegraph-java.jar --src <dir> --out model.json.
| Java construct | kind | traits |
|---|---|---|
| package | package |
TNamed, TModule, TWithChildren (TModule brings definedIn and isStub) |
| class / interface / enum / record / annotation | class/interface/… |
TNamed, TType, TWithInheritances, TWithImplements, TWithChildren, TChildOf, TSourceAnchor, (TComment) |
| method | method |
TNamed, TInvocable, TWithParameters, TWithLocalVariables, TWithInvocations, TWithAccesses, TTypedEntity, TChildOf, TSourceAnchor |
| constructor | constructor |
TInvocable, TWithParameters, … — no TNamed, no TTypedEntity |
| lambda / anonymous class | lambda |
TInvocable without TNamed; disambiguator = (file, startLine) |
| field | attribute |
TNamed, TStructural, TTypedEntity, TChildOf, TSourceAnchor |
| parameter / local var | parameter/localVariable |
TNamed, TStructural, TTypedEntity, TChildOf |
Edges: import (package/type imports → module-level), inheritance,
interfaceImplementation, invocation (via getReferencedTypes()/executable
references), access (field reads/writes), reference (type usages).
Lombok-generated members, if visible: provenance: "generated".
Spoon in noClasspath mode invents plausible FQNs. Rule: build a whitelist
of corpus-declared types first (pass 1); anything referenced but not in the
whitelist becomes { kind: "class", traits: ["TNamed","TType"], isStub: true }.
Never filter by package prefix. Edges to stubs are kept; internal-only
analysis = analyzer-side filter(!isStub).
Decision (M2): isStub is contributed by TModule as well as TType. The
import graph is module-level (§4.6, METAMODEL §9), so import java.util.List
yields an edge to java:java.util — a package no corpus file declares and, with
isStub on TType alone, one nothing could represent. The choices were a class
stub named util (a fabrication), a permanently dangling endpoint (breaking
closure, invariant 10), or making a degraded module expressible. An external
package is now { kind: "package", traits: ["TNamed","TModule","TWithChildren"], definedIn: [], isStub: true } — definedIn: [] is precisely what makes it
external — so "internal view = filter stubs" holds for the import layer too.
Only TType and TModule contribute isStub; a dangling member id is
still refused and reported, because a method stub would need a degraded-member
concept core does not have.
Primitives, void and <nulltype> are not entities. EntityIds.erasedTypeName
keeps primitives as-is by design, so pass 2 omits declaredType for them rather
than letting java:<unnamed>/int reach pass 4 and be fabricated into a stub
class named int. TTypedEntity's value is optional even when the trait is
declared, so omitting is legal and lossless — and a phantom class with a fan-in
of hundreds would distort every coupling metric the analyzer computes.
-
Fixture corpus in
fixtures/java/(overloads, lambdas, inner classes, constructors, static imports, an unresolvable external lib) + snapshotfixtures/java/expected/model.json, pretty-printed and sorted so it is reviewable in a diff.SnapshotTestreproduces it;-Dcodegraph.updateSnapshot=trueregenerates it deliberately. -
Extractor test validates its output against
schemas/model.schema.json(ModelSchemaValidationTest, networknt). -
Profile conformance across the language boundary: a JSON Schema cannot express per-kind trait rules, so
packages/core/test/fixtures-java.test.tsloads the snapshot,parseModels it and runsvalidateModelagainstjavaProfileexpecting zero issues. This is the M2 acceptance gate — the first check that runs both halves of the system against each other. -
CLOSED (M7) — the lambda id disambiguator was too coarse.
Type#file:startLinecould not separate two nameless entities that START ON THE SAME LINE (chain(() -> a, () -> b)), so they shared an id and the first in AST order won: one real entity was absent from the model and nothing distinguished that from an entity never written.**It was worse than a missing entity.** A nameless entity written INSIDE another on the same line took the outer one's id as its `parent`, so the model contained an entity that was ITS OWN PARENT — a containment cycle no analysis walking `parent` survives, and one `validate` could not see. Found on apache/fineract at `json -> gson.fromJson(json, new TypeToken<HashSet<JobParameterDTO>>() {})`: a lambda and an anonymous class, both starting on line 79. The id now carries the COLUMN — `Type#file:line:column`. A column is a source FACT; an ordinal would have made the id depend on the order the extractor happens to walk the AST, so an unrelated change could renumber a corpus. What the corpora recovered: | corpus | entities | edges | |---|---|---| | fixtures/java | 166 → 167 | 173 → 175 | | apache/commons-lang | 15 338 → 15 362 | 24 631 → 24 648 | | apache/fineract | 240 929 → 241 101 | 782 032 → 782 046 | Both corpora still validate clean, and fineract now has zero containment cycles. `StubDisciplineTest.everyNamelessEntityOnASharedLineGetsItsOwnIdentity` replaced the pinned-loss test, as that test's own doc instructed, and `noEntityIsItsOwnParent` guards the consequence. -
Graph closure and self-reference asserted on the snapshot from both sides (
StubDisciplineTestin Java,unknownReferences/selfReferencesin TS). -
Measure resolution rate in noClasspath on a real corpus; report unresolved counts in the extractor's stderr summary. Measured (M2 audit): 100.0% on apache/commons-lang — 263 files, 133 478 type references, 0 unresolved, 15 338 entities (261 stubs), 24 631 edges, 6.6 s wall clock — and 77.8% on spring-petclinic (30 files, 2 189 references, 487 unresolved).
Categorising the failures is what makes those numbers readable, and it first corrected the metric: **every one of commons-lang's 1 466 originally "unresolved" references was `<nulltype>`**, Spoon's static type for the `null` literal — a pseudo-type that can never have a declaration, which the extractor already refuses to make an entity. It is now excluded from the denominator alongside type variables, for the same stated reason. The old 98.9% was measuring how many `return null;` statements commons-lang contains. What is left is a clean statement: the rate measures the corpus's **dependency surface**, not the extractor. commons-lang is self-contained and depends on nothing but the JDK, which resolves against the runner's own classpath — so it resolves perfectly and the 85% target is **vacuous** there. petclinic's 487 failures are Spring and Jakarta types (`jakarta.persistence` 86, `org.springframework.web.bind.annotation` 82, `org.springframework.data.domain` 48, …) whose jars are simply absent; a jar that is not there cannot be resolved by any extractor, so 77.8% is a structural floor for that corpus, not a defect to fix. This is exactly the legacy/non-compilable case M2 exists for, and it is why the stub discipline of §5.2 — not the resolution rate — is the property worth asserting. `ResolutionRateTest` therefore keeps a deliberately low floor plus `unresolved > 0` rather than pinning a number that would pressure someone into deleting the unresolvable half of the corpus. -
Two fabrication bugs found only under real load (M2 audit), both fixed and both now in the fixture corpus (
Batch.java) andArrayAndAnonymousIdentityTest. The fixtures had arrays and an anonymous class, but never readarray.lengthand never touched an anonymous class's own members, so neither was visible: (a) an array type ownslengthandT[]::new; folding those up to the component type produced 8 stub CLASSES namedint,boolean… with a fan-in of 44–88, plus 624 access edges claiming dependencies the source never wrote — the exact phantom §5.2 forbids by name, reaching the model through the one path that never asked the primitive question; (b) Spoon'sOuter$Nname for an anonymous class rendered asjava:pkg/Outer.N, giving one declared class two ids and laundering the second into a stub inside the corpus's own package (invariant 6 failing from the inside). Edges into such a class now name its members precisely.
Input: one or more model.json files (multi-language later — union of models).
- Load & validate against core (profile-aware). Hard fail on schema
errors, collected warnings on profile violations (
loadModels,isClean). - Graph construction: entity map by id; derived inverse indexes
(incomingInvocations, incomingAccesses, subtypes, importers…) computed in
memory, never persisted.
test/no-inverse-index-serialized.test.tsscans every export for an inverse-index key rather than trusting the convention. - Closure check: every edge endpoint / parent / child resolves to a known
id or a stub — asserted as a property, using core's
unknownReferences. - Queries / analyses:
importGraph(module→module, the cross-language layer),typeDependencyGraph,dependenciesOf/dependentsOf,neighboursOf(METAMODEL §9 in one record),coupling(fan-in/fan-out, Ca/Ce, instability) withtopByFanIn/topByFanOut,cycles(Tarjan SCC) at module and type level — each SCC carrying its minimum feedback set and tangle metric (weighted ELS-GR + minimality pass,metrics/tangle.ts; pinned intangle.test.tsand the property suite) — and the view stack (internalOnly,declaredOnly,provenanceOnly,composeViews). - Exports: DOT/Graphviz, CSV (folded graph, coupling, cycles) and JSON artefacts (GraphML/Mermaid later).
Folding aggregates, and keeps what it aggregated. A FoldedEdge carries
count, kinds and provenances; collapsing parallel edges to a bare pair
would discard exactly what makes the result auditable. Self-loops created BY
folding (a method calling a sibling method of its own class) are kept and
flagged — they are cohesion, not the forbidden self-edge of METAMODEL §4.
Self-loops are excluded from coupling by default, on all four counters
together. fanIn/fanOut AND outgoingEdgeCount/incomingEdgeCount obey
includeSelfLoops as a unit, so a row can never read "fanOut 0, outgoing weight
6". Counting a self-loop would give every cohesive class Ce ≥ 1 and Ca ≥ 1 and
instability could never reach 0 or 1. The conservation law follows per mode:
with includeSelfLoops: true the row sums equal diagnostics.foldedEdges
exactly; by default they equal the non-self folded weight. Both are pinned.
Tarjan is iterative, with an explicit frame stack. The recursive formulation exhausts V8's stack at the ~15 000 nodes commons-lang folds to, and it fails only on real corpora because every hand-built test graph is shallow. Guarded by a 20 000-node chain and a 20 000-node ring.
A stub is its own container at every fold level. A stub has no parent, so a
stub CLASS becomes its own module node. Consequence, measured on the fixture:
24 of the 27 module-level nodes are external classes. Anything user-facing
should default to internalOnly at module level. See the M4 note below.
An analysis number without its view is not a fact. FoldedGraph,
CouplingTable and CycleReport all carry level and view, and every CSV
row repeats them as columns (a # comment line is data to an RFC 4180 parser).
- KNOWN GAP carried into M4 — stub classes do not fold into their stub
package. The extractor emits stub packages (
java:java.util) and stub classes (java:java.util/List) as unrelated roots, so the unfiltered module graph lists external CLASSES as modules. The import layer is unaffected (the extractor already writes imports module→module:nonModuleEndpointsis 0 on the fixture, gson and commons-lang), but the full type-fold at module level is noisier than it should be. The fix is aparenton stub classes in the extractor, not a special case in the fold.
✅ Shipped in M4. Every command takes one or more model paths and loads them as a union (decision 5).
codegraph validate <model.json...> [--json]
codegraph analyze <model.json...> --report deps|cycles|coupling [--level module|type]
[--internal-only] [--declared-only] [--json] [--top N]
codegraph export <model.json...> --format dot|json|csv|plantuml [--level module|type]
[--internal-only] [--declared-only] [--out FILE]
codegraph profiles [--lang java] [--json] # print a profile spec
-
validate— runs the analyzer'scheckConformancegate over the union. -
analyze— deps / cycles / coupling, at module or type level. -
export— DOT, JSON, CSV and PlantUML (class diagram) of the folded graph. The PlantUML rendering keeps the DOT encoding: solid arrow = all-declared, dashed = contains an inference, label = folded count,<<stub>>= external. -
profiles— prints core's profile data; synthesizes nothing. -
--helpper command,--version, and a usage error naming the valid values for a bad flag.
Locked behaviour, all covered by the end-to-end binary suite:
- Exit codes:
0ok ·1internal bug ·2usage ·3findings. 1 and 3 are never conflated, so a CI job can gate on model quality alone. - Streams: stdout is the artifact ONLY; every warning, summary and fold diagnostic is stderr. Verified by real shell redirection, not in-process.
- No ANSI, ever. Deterministic: identical inputs give byte-identical stdout.
--jsonon every reporting command, always a self-describing object with akindstamp — the same facts as the text form, differently printed.
✅ Shipped in M4 as checkConformance (in the analyzer, so the CLI, the property
suite and CI all ask the same question), plus a fast-check property suite.
Invariants from the design doc, run against every extractor output:
- Closure: no edge to an unknown id (stubs count as known).
- No self-reference:
from !== toon every edge. - Provenance always set;
candidatesnon-empty when present, and present only ondynamic-candidateedges. - Profile validity: every entity passes
validateEntity. - Anchors: every edge anchored; spans 1-based, ordered.
- Duplicate ids: identical redeclaration is legal (§1.1); only a disagreement on kind or trait set is an error.
- Determinism: two runs on the same corpus produce identical output.
- Generative side: arbitrary entities from a profile always round-trip JSON → validate → JSON.
Not asserted, deliberately: that to appears in its own candidates list.
§4 calls to the "best candidate", but measured against real Spoon output
(commons-lang: 361 dynamic-candidate edges) 119 correctly EXCLUDE it — those
resolve to an interface or abstract method, which cannot itself run, so the
candidates are the concrete overriders. The model records no abstractness, so
nothing can distinguish a correct exclusion from a mistaken one; re-instating
the rule requires an abstractness fact in the metamodel first.
Cross-validation strategy (later, when a 2nd Java extractor exists, e.g. Tree-sitter): Spoon output is the oracle; property = Tree-sitter edge set ⊆ Spoon edge set; any gap = a missed resolution case.
Motivation (M4 aftermath): extracting apache/fineract produced a 559.5MB
model.json — over Node's string ceiling (0x1fffffe8 ≈ 512MB), so the
analyzer cannot read it at all. Profiling showed ≈76% of the bytes are
repeated strings (edge endpoint ids 186MB, anchor paths 117MB, entity ids +
parents 88MB) with tiny value sets. Design docs, which this phase executes:
docs/model-metamodel.md (MM-1…MM-5, format-independent) and
docs/model-encoding.md (JSONL contract + SQLite cache). Clean break
in place: schemaVersion stays "1.0.0" — the format is entirely
internal for now, there is no external consumer to negotiate with, so the
contract changes under the same version and old files are simply
regenerated. No old-format reader (decision below).
The three milestones are sequenced so every one lands green: M5 adds v2 concepts to core while v1 keeps building; M6 is the breaking change; M7 is additive on top of M6.
✅ Shipped. The wire format is untouched (schemas/ regenerated to a byte-identical
file); what changed is what the vocabulary MEANS, so M6 has something to encode.
- METAMODEL.md amendments: §1.1 restated — identity is the structured key
(lang, module, symbol, disambiguator?), rendered ids display-only (MM-1); §4 extended to nameparent/children(MM-2); §5.1 vocabularies as closed referential sets (MM-3); §5.2 profile validity as a function of(kind, trait set)(MM-4); §8 split into 8a model vs. 8b encodings (MM-5). CLAUDE.md invariants 4 and 7 restated to match. -
core:packages/core/src/identity.ts—NaturalKey(type + Zod schema),renderId,naturalKeyIndex,naturalKeysEqual,compareNaturalKeys/sortByNaturalKey(canonical order),duplicateNaturalKeys,naturalKeyIssues. -
core:validateEntitysplit into a memoized(kind, trait set, isStub)verdict and the per-entity value checks (MM-4). - Property suite: natural-key uniqueness generatively, plus rendering injectivity and canonical order as a total order.
- DoD met: METAMODEL.md v2 merged; core exports the identity vocabulary;
the entire v1 suite still green (1 089 TS tests,
pnpm -r buildclean,./mvnw packageclean,schemas/unchanged).
Decision (M5) — separators are reserved so rendering is injective. A key
whose module contained /, or whose symbol contained #, would render as
some other key's id, and the two entities would silently merge — the M2
archive(List) collision (§4.6) one level up, where it again costs a whole
entity with no error. renderId therefore validates and throws rather than
rendering a lossy id. Pinned by an exhaustive test over an alphabet built from
the separators themselves plus their concatenations: an alphabet of unrelated
words let a deliberately broken renderer pass, which is why the generated
pieces are a, b, a.b, a/b, ab.
Decision (M5) — a module names ITSELF in the module component, with an
empty symbol, rather than referencing its parent module as the design doc's
MM-1 first drafted. The parent form renders java:com.acme.order as
java:com.acme/order, breaking the frozen id scheme, and needs a fabricated
java module to place the stub package java:java.util whose parent no corpus
declares — the fabrication METAMODEL §6 exists to prevent. Consequence for M6:
a package record's m surrogate points at its own row.
Note — MM-4's memo key carries isStub and the profile. Validity is a
function of (kind, trait set) only within one profile and one side of the
stub exemption (§6 waives the lower bound for stubs). Both bounds are pinned by
tests that fail if the key is narrowed.
✅ Shipped.
-
core: record schemasHeaderRec/FileRec/EntityRec/EdgeRec/EofRecdiscriminated ont(wire.ts); streaming codec (jsonl.ts) and file I/O (jsonl-file.ts);childrenkey removed fromEntity(TWithChildrenstays a marker trait); traits ride as int arrays into the header'sdict.traits; everyEntityId-typed field is a surrogate int, driven byENTITY_REFERENCE_KEYSso a new referencing trait cannot be forgotten. -
gen:schemas: one JSON Schema per record type + a GENERATED container contract (schemas/README.md) for what a single line cannot express — section order, dict resolution, trait keys, closure, canonical order. Its trait/key table is printed fromWIRE_TRAITS, so the published rules cannot drift from the code that enforces them. -
extractors/java:NaturalKey+ canonical sort → surrogate assignment → streamingJsonlWriter(one Jackson generator, one line at a time);eoftrailer carries counts. -
Property suite reformulated over surrogates: closure as
ref < entities countchecked as references resolve; truncation detection (every proper prefix must be refused); determinism as byte-identical output whatever order the extractor emitted. -
Fixtures regenerated to
.jsonl; the v1 document path andschemas/model.schema.jsondeleted. -
CLI reads
.jsonlthrough a CHUNKED SYNC reader — the ceiling is one JavaScript string, not sync I/O, so the CLI stays synchronous end to end instead of every command becoming async to buy nothing. -
DoD met, measured:
corpus v1 v2 commands fixtures/java 155KB model.json41KB model.jsonl(3.8×)all apache/commons-lang 17.1MB 6.9MB (2.5×) all, < 1 s each apache/fineract 559.5MB — unreadable 127.4MB (4.4×) all, 11–12 s, ≤ 1.6GB RSS Fineract extracts in 94 s (240 929 entities / 782 032 edges, 0 edges dropped) and
validatereports a clean bill of health. The 127MB is short of the ~90MB the encoding doc estimated — the estimate assumed more of the file was id strings than it is — but the number that mattered was never the ratio: v1 could not be READ at any size, and now it can.Regression check on commons-lang: the v2 model has the same 15 338 entity ids and the same 24 631 edges as the v1 extraction, byte-for-byte identical content modulo the two intended metamodel changes below. The format changed; what the extractor claims did not.
Decision (M6) — closure is enforced at WRITE time, not reported after. A reference is a surrogate into the file's own entity section, so "points at nothing" is not expressible. Consequences, all deliberate:
- the extractor gained a pass 4.5 that DROPS an edge whose endpoint nothing declares, and counts it on stderr (0 on the fixtures, commons-lang and fineract) — v1 wrote such an edge and left the analyzer to report it;
- a dangling reference in a FILE is now a malformed file (a schema error),
not a finding about a valid one.
LoadDiagnostics.danglingReferencessurvives for models built in memory, and the CLI still exits 3 either way; - cross-model references are gone too: each model is closed on its own, and a reference to another language's entity is a STUB that the union merges by natural key. That is the stub discipline of METAMODEL §6, applied to the multi-language case.
Decision (M6) — a stub's MODULE is materialized even when its PARENT is refused. Identity names a module (MM-1), so an entity whose module is not declared cannot be written at all; containment is a separate claim, and §6.1's two refusals (a corpus-declared package, the unnamed package) still stand. The two questions were conflated before because nothing forced them apart.
Bug found by fineract, fixed — the corpus whitelist claimed types the corpus
never wrote. CorpusWhitelist walked every CtType Spoon exposed, including
the SHADOW types noClasspath materializes for unresolvable references
(jakarta.ws.rs.core.MediaType, java.math.BigDecimal, …). The entity pass
correctly refused them — no source position, so no evidence — and the stub pass
then refused to degrade them ("declared by the corpus but never emitted"),
leaving every reference to them dangling. Under v1 those became dangling
references; under v2 the model could not be written. The whitelist now asks the
same "is it written here?" question the entity pass asks, which is what its own
contract already demanded.
Bug found by fineract, fixed — entities.push(...model.entities). A spread
passes one ARGUMENT per element, so the union overflowed the call stack at
240 929 entities and validate reported an internal error. It fails long before
memory does, and no test-sized graph reaches it; packages/analyzer/test/scale.test.ts
now unions a 200 000-entity model.
Bug found by MM-2, fixed — executables never declared themselves containers.
A method's parameters and locals carry parent, but method/constructor/
lambda carried no TWithChildren and no children, so METAMODEL §3.2's "both
stored, must agree" was false for every method with a parameter. v1 could not
see it: the check only looked at entities that HAD a children key. The Java
profile now licenses TWithChildren on the three executable kinds and the
extractor emits it (4 795 entities on commons-lang).
Work in progress. Design settled by a scout-and-judge pass over the code
(4 readers → 3 independent proposals → one synthesis); the recommendation is
cache the parse, not the graph — model.db replaces the JSONL decoder and
nothing above it, so byte-identity is arithmetic rather than vigilance.
Measured on apache/fineract, which is why:
| stage | time | |
|---|---|---|
| decode JSONL | 7.1 s | the dominant cost |
parseModel (Zod over the whole Model) |
2.6 s | redundant — the decoder already validated every record |
validateModel (profile) |
1.3 s | real work |
unknownReferences (closure) |
0.8 s | redundant since M6 — a dangling surrogate is unreadable |
buildGraph |
0.6 s | real |
| fold + coupling + cycles + export | 0.2–0.6 s each | real |
≈93% of a command is decode-and-revalidate; the analysis itself is under a second. Pushing metrics into SQL would optimise the 6% that is not the problem.
Steps landed:
-
Step 1 — split parse from unify (
12c07f6).loadDecodedModelsruns the unify and diagnose halves only;loadModelskeeps its signature asparseModelthen that, for raw JSON and in-memory models. The CLI reads through core's decoder, so it uses the new entry point. Profile validation, closure over the union, self-edges and cross-model redeclaration all still run —load-equivalence.test.tspins that both entry points agree, and catches divergence (verified by mutation). fineractanalyze: 11.50 s → 9.13 s, 1 438 MB → 1 099 MB, all 32 baseline outputs byte-identical. -
Step 2 — freeze the two order contracts, on pre-DB code, because the store is what will break them. -
fold-order-contract.test.ts: no rendering depends on the order edges ARRIVED in. AFoldedEdgeaggregateskinds/provenancesinto Sets, and a Set iterates in insertion order; today that is the decoder's canonical order, and a query planner's order is not. Asserted over the real pipeline at both levels and under a view, for DOT, PlantUML, CSV, JSON, cycles and coupling — and pinned as non-vacuous, since a single-member Set has no order to get wrong. Unsorting any of the four exporters fails it. -fixtures/unicode/+collation-contract.test.ts: canonical order is by UTF-16 CODE UNIT (compareIds), SQLite'sBINARYcollation is by UTF-8 BYTE. They agree across the BMP and invert above it —𠀀…(U+20000) leads with code unitD840but byteF0, whileA…(U+FF21) leads withFF21butEF. Both are legal Java identifier letters, so it is a corpus someone could write. The committed reference report is the artefact every later step is diffed against.**The rule both contracts state: sorting belongs to the model, never to the storage engine.** A query needing canonical order sorts in TypeScript or `ORDER BY`s a column the importer wrote in canonical order. -
Step 3 — expose core's validated wire-record stream.
ModelDecoderdid two jobs: validating the wire, and materializing aModel. They are nowRecordReaderandModelBuilder, andreadModelRecordsSync(path)yields validated records without building anything — the importer writes rows as they go by and never holds the corpus.readModelFileSyncis a consumer of that same stream, which is what makes the two readers ONE reader rather than two that drift.The trait-key rule moved from materialization into the reader, so it is now reported with its line number and holds for every consumer. `record-stream.test.ts` states the property as equivalence: nine broken files — not JSON, not a record, truncated, miscounted eof, dangling edge surrogate, missing trait key, forward module reference, empty, headerless — must be refused by BOTH routes with the identical message on the identical line. Plus the case that motivates the whole step: a truncated file is still a sequence of perfectly good JSON lines, so a hand-rolled `JSON.parse`-per-line importer accepts it happily and stores a corpus silently missing its tail. Measured on fineract (241 101 entities / 782 046 edges), separate processes: **stream 4.6 s at 140 MB peak; materialize 6.8 s at 555 MB.** Validating costs a quarter of the memory of keeping. -
Step 4 — one load site for Node's SQLite builtin.
loadSqlite()inanalyzer/src/store/sqlite.tspatchesprocess.emitWarning, loads viacreateRequire(import.meta.url), restores in afinally, and memoizes. Three independent reasons it cannot be a static import, in increasing severity:1. ESM evaluates every `import` declaration before any statement of the importing module body, so the ExperimentalWarning is already on stderr by the time a patch could exist — and several CLI end-to-end tests assert `stderr === ""` on the built binary. 2. `await import()` loads late enough but is async, and the read path is synchronous end to end; one `await` would reach `main`. 3. esbuild rewrites a static `from "node:sqlite"` to `from "sqlite"` in the tsup output, stripping the `node:` prefix. Harmless for most builtins — `node:fs` and `fs` both resolve — and fatal for this one, which is prefix-only. **Verified by mutation: the built CLI then does not start at all.** The specifier inside `require()` is opaque to the bundler and survives verbatim. The returned `SqliteApi` is written from the store's needs (`exec`, `prepare`, `run`/`get`/`all`/`iterate`, `close`) rather than re-exporting the builtin's types, so PLAN's `better-sqlite3` fallback stays a matter of satisfying one interface at one site. `open()` passes `options ?? {}`: the builtin validates by ARITY, so an explicit `undefined` is not an omitted argument and throws. Two tests, because neither sees the other's failure. `store-sqlite.test.ts` spawns processes — stderr is only observable in one — and asserts the load is silent, that a static import of the same builtin is NOT silent (the control, so the guard cannot die quietly on a future Node), and that `emitWarning` is restored identically. `source-hygiene.test.ts` asserts that exactly one source file in the workspace names the builtin at all, which is the failure a stderr assertion cannot see: a second, equally quiet loader free to drift from the first. Also stated against the real engine for the first time: `ORDER BY` under BINARY collation really does return the unicode fixture's ids in a different order than `compareIds`, and ordering by the surrogate the importer assigned restores it. Step 2 proved that with `Buffer.compare`, a model of SQLite; this proves it with SQLite. -
Step 5 — freeze the DDL, against M6 rather than the sketch.
analyzer/src/store/schema.tsholds the schema, the open options, themetakey set, and the key→storage mapping. Four things the M6 wire changed about docs/model-encoding.md's earlier sketch:1. `module_id INTEGER **NOT NULL**` — `m` is required on every entity record and a module names itself, so the sketch's "NULL for root modules" case no longer exists. 2. `entity_trait(entity_id, trait_id)` → an interned `trait_set`. Measured: **15 338 entities / 15 distinct trait sets** on commons-lang, **241 101 / 18** on fineract, so the join table would carry ~1.3M rows to say 18 things. Size is the smaller half — MM-4 makes profile validity a function of `(kind, trait set)` and core's validator already memoizes on that key, so the interned id IS the key. `ord` is kept so an entity's `tr` array re-encodes exactly. 3. **No UNIQUE natural key and no `CHECK (to_id <> from_id)`.** Duplicate identities and self-edges are conformance FINDINGS; a store that refused to cache them would make `import` fail on exactly the models `validate` exists to report on. Same rule as step 3: two gates on one format drift. 4. Array-valued keys get ordered tables (`entity_comment`, `entity_defined_in`, `entity_parameter`, `entity_local_variable`, `edge_candidate`) rather than JSON, so they stay joinable; `extra` holds only keys `core` does not type, since records are loose by design. Two things the implementation turned up that reasoning had not. **Foreign keys are a decision, not a default:** SQLite's default is OFF and Node's builtin overrides it to ON, so `STORE_OPEN_OPTIONS` states it — off, because `parent`/`declaredType` legitimately point forward and immediate constraints would reject valid models, while closure is already the reader's guarantee. `PRAGMA foreign_key_check` still audits a suspect cache, which `foreign_keys = ON` would not. And one addition to `core`: **MM-3 says a vocabulary is a SET**, so `RecordReader` now refuses a header dictionary that repeats an entry — a repeat makes two indices name one thing, which any interning store breaks on. It is a property of the file, so it belongs in the reader; stated there, both read routes refuse it identically. `store-schema.test.ts` (17) asserts everything against a REAL database created from the DDL, never against its source text: a `sqlite_master` snapshot of the effective schema; the invariant-4 guard (`edge_to` and `entity_parent` are indexes, no table name reads as an inverse); the "caches what the reader accepts" rule as behaviour, by inserting a duplicate natural key and a self-edge; and the drift guard — every key `WIRE_TRAITS` and the record schemas can carry maps to a real column of a real table, so adding a trait key to `core` fails the store until someone decides where it goes. Five mutations, each caught by the intended test — and S3 (making the natural key UNIQUE) exposed a defect in the test itself: the duplicate it inserted differed only by a NULL disambiguator, and SQLite treats NULLs as distinct in a unique index, so it would have passed either way. Now the duplicate carries a disambiguator, with the NULL case asserted alongside.
Remaining steps (from the synthesis, unchanged):
Role split (encoding doc §1): .jsonl is the CONTRACT — schema-validated,
diffable, language-agnostic; model.db is the WORKBENCH — a derived,
disposable cache, regenerable at any time, never committed, never the
interchange.
-
Step 6 —
importModel(jsonlPath) → model.db, and its exact inverse. Driven byreadModelRecordsSync, one transaction, prepared statements; the corpus is never held.readStoreRecords(db)yields the wire records back out, so losslessness is an EQUALITY rather than a checklist:[...readStoreRecords(db)]must equal[...readModelRecordsSync(path)], record for record. 1 029 842 records identical on fineract.hydrateModelis that stream fed to core's ownModelBuilder— the same onereadModelFileSyncuses, so a model from the cache cannot diverge from one off disk.The acid test earned its name on the first run, finding two real bugs of one kind: **an empty array and an absent key are the same rows.** `definedIn: []` writes nothing to `entity_defined_in`, exactly like an entity with no `TModule` — and the fixture has a stub module that proves it. Presence now comes from the TRAIT SET, which is where the wire keeps it (`TComment` contributes `comments`, so carrying the trait IS carrying the key). `candidates` is the one array-valued key no trait contributes, so `edge.candidate_count` records its presence: NULL absent, 0 empty. Measured on fineract (241 101 entities / 782 046 edges): | | time | peak RSS | |---|---|---| | import (once) | 9.0 s | 289 MB | | read the model from `.jsonl` | 6.4 s | 701 MB | | hydrate the model from `.db` | 3.6 s | 592 MB | | fan-in top 20, in SQL | 0.04 s | — | **The finding that redirects step 7: the win is not hydrating faster, it is not hydrating at all.** Full hydration was first measured at 7.6s — SLOWER than reading the text — and only beats it after switching the bulk queries to `setReturnArrays(true)` (1 023 147 rows: 3.83s as objects, 1.46s as arrays). Even at 3.6s that is a 1.8× constant, where the same question asked in SQL is 0.04s against 6.4s. A DB-backed facade that begins by rebuilding the whole `Model` gives up the actual win. Two smaller decisions, both measured: indexes are created AFTER the inserts (10.75s → 9.0s, 178.4MB → 169.8MB), and atomicity comes from a rename rather than from the journal, which is what makes `journal_mode = OFF` safe. Four mutations. Dropping `space`, inferring list presence from rows, and ordering entities by `symbol` instead of by surrogate were all caught. The fourth — writing straight to the target with no temp file — was NOT, because deleting the target on failure also "leaves no database". The property that separates them is the one users feel: a failed re-import must not cost you the cache you had. Now asserted. -
Step 7 — fold in SQL.
foldFromStore(db, options)answers in SQL whatfoldGraphanswers in JavaScript. Fold is the FUNNEL — coupling, cycles, DOT, PlantUML, CSV and JSON all consume aFoldedGraphand look at nothing else — so this is the one place worth moving, and everything downstream gets it unchanged.Measured on fineract, all eight (level × view) combinations in parity: | | in memory | from the store | |---|---|---| | prepare (build graph / open) | 9.0 s | ~0 s | | fold, module level | 0.4 s | 1.4 s | | fold, type level | 0.5–1.0 s | 1.5–1.6 s | | **total** | **9.4 s** | **1.4 s** | Stated honestly: the SQL fold is SLOWER per call — the win is entirely in not needing the 9s graph build, so a caller that folded many times would amortize the other way. Each CLI invocation folds once. The 782 046 base edges never enter JavaScript; only the ~20 000 aggregated ones do. Containers are resolved by a FIXPOINT over `parent_id`, not a recursive CTE: measured 0.38s against 1.06s, because the CTE re-walks each chain from every descendant. The loop ends when a pass adds nothing, so a malformed parent cycle terminates — its members just never resolve, which is what the in-memory walk also does. **It refuses rather than approximates.** A view is an arbitrary predicate pair; SQL cannot run one. `translateView` translates the ones a view NAMES through `ViewDescriptor.filters` (`internalOnly`, `declaredOnly`, `provenance:…`) and returns `undefined` for anything else, so the caller hydrates. A facade that quietly answered a slightly different question would be worse than none: the numbers would still look like numbers. `assembleFoldedGraph` was extracted from `foldGraph` so both producers sort, index and freeze through one function — the step-3 rule applied to a second producer. `store-fold.test.ts` (22) compares whole graphs — nodes, edges, counts, kind/provenance sets, diagnostics — for both levels × five views, plus self-loop dropping, edge-kind filtering, and the unicode fixture. Then it proves the payoff rather than assuming it: DOT, PlantUML, CSV, JSON, coupling and cycles byte-identical from either fold. Four mutations; two exposed gaps rather than confirming coverage. Silently accepting an untranslatable view, and miscounting `droppedEdges`, were caught. Restricting the container walk to view-included entities was caught only after being rewritten to break chains DURING resolution — the first version deleted already-resolved rows and was a no-op. And deciding "is this its own module?" by comparing SYMBOLS instead of surrogates passed everything: no fixture had a type whose symbol equals its module's path. That case now exists, because getting it wrong renders a class as its own package — two entities, one id. -
Step 8 — diagnose from the store.
diagnoseStore(db)returns theLoadDiagnosticsloadDecodedModelsreturns, computed by query rather than by walking entities. The store is not asked to REMEMBER a diagnosis — a cached verdict inside a cache is a second thing to invalidate — it is asked to answer one.**MM-4 is what makes it cheap.** Profile validity is a function of `(kind, trait set, isStub)`, and the store interns trait sets, so `SELECT DISTINCT kind_id, trait_set_id, is_stub` is 23 rows over fineract's 241 101 entities. Each verdict comes from `entityCompositionVerdict` — now exported from core, the same memoized function `validateEntity` calls — and a verdict with no issues needs no entities at all, so a clean corpus enumerates nothing. `edge-kind-not- allowed` is likewise a function of six values, not 782 046 edges. Measured on fineract: **0.46 s from the store against 2.10 s in memory, and that 2.10 s needs a 6.21 s read first** — 8.3 s → 0.46 s. One check is skipped and says so: the per-entity Zod parse of each declared trait's keys, which is the bulk of the in-memory cost. It cannot fail on a model that reached a store, because the record reader enforced `WIRE_TRAITS` at import. That is an argument, so it is a test — every trait's key set is pinned to match between `TRAITS` and `WIRE_TRAITS`, and a value the model schema rejects is shown to be refused on the wire. Self-edges are the one diagnostic needing whole `Edge` objects, so when one exists the model is hydrated and core's `selfReferences` answers. A self-edge means the corpus is already broken; paying a hydrate there costs nothing on every correct run, and beats a second edge constructor drifting from `ModelBuilder`'s. **Both real corpora are clean, so parity on them proves almost nothing** — returning "no issues" unconditionally would pass. Every category is therefore exercised on a model broken on purpose, each asserting the issue exists before asserting the two paths agree about it. Four mutations; two exposed gaps, and building the case for a third found a real bug. Comparing interned `trait_set_id`s instead of the canonical trait SET was caught (two declarations differing only in trait ORDER are the same declaration). Dropping the emission order was NOT — until a model existed with issues from two categories landing in the opposite order. Constructing it surfaced the bug: `isStubEntity` is "declares TType or TModule AND isStub is true", and reading the `is_stub` column alone calls an entity a stub that the in-memory path does not — which flips the composition verdict, since the trait lower bound is waived for stubs. And the unknown-profile branch was returning no `duplicateIds`, where the in-memory path computes them regardless of profile; the test passed only because no fixture had a duplicate. -
Step 9 —
codegraph import. The explicit command:codegraph import <model.jsonl...> [--out FILE] [--json]. It is unconditional — the user asked for a store, so they get a fresh one, and nothing here guesses whether the old one was still good. That question belongs to the automatic cache, which has to answer it unasked.**One model per store.** Every other command unions its positionals; this one deliberately does not, because surrogates are file-scoped and are not identity (MM-1), so a union would mean renumbering — and a renumbered corpus is a repointed one. Several paths mean several imports, and `--out` with more than one is a usage error rather than a silent choice. Two decisions the CLI's own contracts forced: - **stdout says what is true of the STORE; stderr says what was true of the RUN.** SQLite promises no byte-determinism (page allocation varies) and a duration never could, so the size and the timing must not reach the stream `e2e-determinism` compares. - **A file that is not a model is a FINDING, not a crash.** Left unhandled, the record reader's `JsonlError` escaped to `main` and was reported as "an internal error — a bug in codegraph", blaming the tool for the user's file. It now exits 3 with the reader's own message, and one run reports every bad path while still importing the good ones — the same choice `loadModelFiles` makes for the reading commands. Exit 3 also covers a model that reads fine but breaks its profile: the store IS written and usable (refusing to cache what `validate` exists to describe would be the second gate this project keeps declining to add), and the exit code still says something is wrong. Step 8 makes that check affordable — `diagnoseStore` on fineract costs ~0.5s, not 8s. Also fixed: the 20 000-entity round-trip from step 6 was FLAKY. It takes ~0.4s alone but exceeded vitest's 5s default under the contention of `pnpm -r test`, where several packages' workers share one machine. A test that fails on a busy CI box and passes on a quiet one teaches nobody anything, so its budget is now stated. -
Step 10 — the automatic cache.
analyzeandexportgiven a.jsonlbuild the sibling.dbif they must, reuse it if they can, and answer from it.--no-cachereads the model directly.Measured on fineract, `analyze --report deps`: | | wall | peak RSS | |---|---|---| | first run (builds the store) | 14.9 s | 268 MB | | **second run (reuses it)** | **1.9 s** | **163 MB** | | `--no-cache` | 11.8 s | 1 094 MB | **6.3× faster and 6.7× less memory**, and the 19 090-line report is byte-identical to the one the model produces. `AnalysisSource` (cli/src/source.ts) is the seam: neither command branches on where the answer came from, so there is no second formatting path to keep in step. The cache is used only when it can be EXACT — one model (a store holds one; a union would mean renumbering), no `--no-cache`, a writable store, and a view SQL can translate. Every other case falls back to what was there before, so the worst case is the old speed, never a different number. `importGraphFromStore` was added so `deps --level module` — the most-used report and the one cross-language layer — is not the single command that has to hydrate. Three things this found, two of them real bugs: 1. **`open: false` does not mean "do not create".** In Node's SQLite it means "construct the handle but defer opening", so every freshness query threw, every store read as unreadable, and **the cache rebuilt itself on every run while looking like it worked.** Only the determinism suite noticed — via the elapsed time it printed. 2. **A model the reader refuses was crashing.** `openCache` rethrew `JsonlError`, which reached `main` as "an internal error — a bug in codegraph", blaming the tool for the user's file. A refused model is now simply a reason to have no cache; the reading path reports it as a finding, with the reader's own message and exit 3. 3. **stderr is a diffed stream.** `e2e-determinism` says so and explains why — an elapsed-time figure turns every CI log into a false diff — so the cache note is one line, `cache: <path>`, identical whether the run built the store or reused it. Whether *this* invocation paid for the import is a performance detail, and `codegraph import` is the command that reports it. Staleness is size and mtime, not a content hash: hashing 133 MB costs more than the parse the cache exists to avoid, which would make the check more expensive than the miss. The trade is stated rather than hidden, and both halves are tested — a changed model AND a model rewritten to the same size with a later timestamp. `*.db` is gitignored: the store is derived and disposable, and a binary beside the `.jsonl` in git would be a second, undiffable answer to the same question. -
Step 11 — the SQL cookbook.
docs/sql-cookbook.md: orientation, two setup views that turn surrogates into rendered ids, and recipes for fan-in, fan-out, facts-only, traits, modules, the import graph, reachability, module cycles, evidence and uncertain dispatch — plus the four rules a query must respect and the list of questions to askcodegraphinstead.**The cookbook is executable.** `sql-cookbook.test.ts` extracts every ```sql block and runs it in document order against a store built from the committed fixture, because a recipe that has rotted still LOOKS like SQL and the reader finds out at a prompt rather than in review. Three claims the document makes about agreeing with `codegraph` are checked as equalities, not for syntax: the `node` view reproduces `renderId` for all 167 ids in order, the fan-in recipe counts what the model's edges count, and the cycle recipe agrees about cycles. Two mutations. Breaking a recipe fails three tests. **Rendering the self-module test by SYMBOL instead of surrogate did not** — the same hazard that survived step 7's parity suite, now living in the documentation, where it would have taught the wrong idiom. The namesake corpus is now built here too. Written while checking rather than asserting: `module_id` and the fold's container walk agree for every entity that resolves on both corpora (0 disagreements), but they differ where the walk ends with NO container and `module_id` always points somewhere — the fixture has 2 such entities. The document says so rather than implying they are the same question.
Every clause verified on apache/fineract (241 101 entities / 782 046 edges), not asserted.
| DoD clause | evidence |
|---|---|
a second analyze opens the cache without re-parsing |
cold 10.9 s / 241 MB → warm 1.96 s / 114 MB; --no-cache 9.8 s / 1 053 MB |
| every report byte-identical to its JSONL-only output | 12 of 12, cmp-verified: 8 analyze variants and 4 export formats, 486 000+ lines including deps --level type at 124 266 lines and export --format json at 134 077 |
| ad-hoc SQL cookbook documented | docs/sql-cookbook.md, every query executed by a test |
The cache is 5× faster and 9× lighter on the report that motivated M7, and
the whole M6 → M7 arc takes analyze on fineract from 11.5 s / 1 438 MB to
1.96 s / 114 MB — 5.9× faster on 12.6× less memory, with the answer
unchanged to the byte.
What the eleven steps actually bought, in order of how much it mattered:
- The wire became streamable (steps 1–3, 6). A 559.5 MB v1 model was unreadable at all — past Node's single-string ceiling. It is 133.7 MB of JSONL now, and validating it costs 140 MB of memory where materializing it costs 555 MB.
- The fold moved into SQL (step 7). Fold is the funnel every report and export goes through, so moving that one thing moved all of them, and the 782 046 base edges never enter JavaScript.
- MM-4 made validation nearly free (step 8). Profile validity is a
function of
(kind, trait set, isStub)and the store interns trait sets, so 23 rows decide a 241 101-entity corpus: 2.1 s → 0.46 s. - The store answers, it does not remember (steps 5, 8). No cached verdicts, no second representation to invalidate — every diagnostic is a query.
And what the process bought, which is most of why the numbers are trustworthy:
of roughly two dozen mutations, five exposed gaps rather than confirming
coverage, and three of those found real bugs that no test had asked about —
open: false silently rebuilding the cache on every run, an empty array
indistinguishable from an absent key, and is_stub read without the trait that
gives it meaning. The two that recur are worth naming: a test that passes
because the fixture has no instance of the case (the namesake type, the
duplicate under an unknown profile), and a mutation written so symmetrically
that it cannot be detected by construction.
- C# via Roslyn — planned in full as Phase 10 (§13, M12): the second real extractor, first-class on Linux/macOS, shipped as one binary per OS.
- Clojure via
clj-kondo --analysis→ thin JSON adapter (near-free; first cross-language test on the Import layer; exercises the fn-var case for real). - TypeScript via the TS compiler API — planned in full as Phase 11
(§14, M13): self-hosting (run codegraph on codegraph),
space: type|value, declaration merging, and thenamespace-style legacy corpus. - SCIP adapter (one effort → Rust + TS + Python via existing indexers).
- Code city visualization (
packages/viz, Three.js + Vite): render the analyzed model as a 3D city — districts = modules, buildings = types (height/footprint/color mapped to documented metrics), edges as flows. Consumes analyzer output only; visual-language rules live inCLAUDE.md. - Elixir via the compiler's parser as a library plus a lexical resolver,
compilation tracers as the optional enrichment — planned in full as
Phase 13 (§16, M15): the first macro-first, dynamically dispatched
language,
generatedanddynamic-candidateprovenance exercised for real.
Documented static limits (all languages, per profile notes): reflection,
Class.forName/Spring XML, service loaders, pre-expansion macro code.
Motivation: model.jsonl is a snapshot — it says what the code IS, not what
happened to it. The crime-scene analyses (hotspots, logical coupling,
knowledge maps — Tornhill, Your Code as a Crime Scene) and a Gource-style
replay of the city both need the time axis. Two prior decisions make this
phase cheaper than it looks: identity is the natural key (invariant 7), so
"the same entity across two snapshots" is key equality — no diff heuristics
for named entities; and Spoon runs noClasspath, so historic commits that no
longer compile still extract.
Three principles, locked up front:
- Evolution facts are a third artifact.
history.jsonlsits besidemodel.jsonl: repo-scoped, language-agnostic, and changing on every commit while structure does not. It never merges into the model file and never grows a fifth provenance value — code facts and history inferences stay unmixable. The join happens in the analyzer, onanchor.file/TModule.definedInpaths relative to the analyzed root. - The miner has no code intelligence — the extractor rule, mirrored.
It emits paths, authors, timestamps, and line deltas from one
git logpass; everything smarter is derived downstream. - Lineage v1 is the natural key, exactly. A renamed symbol is a death
plus a birth; anonymous entities (
file:line:columndisambiguators shift under any edit above them) are not tracked over time. Both restrictions are documented, not silent.
Everything Gource shows comes from git log alone — so the replay
experience ships before any extractor touches the time axis, and the one
genuinely hard viz problem (layout stability) is forced on cheap data.
-
codegraph scm <repo> [--since <date>] --out history.jsonl(scm, notgit: the door stays open for hg/fossil): onegit log --numstat --no-merges --find-renames -zsubprocess, parsed by pure@codegraph/scm. M6 discipline reused: header dictionary (author table), interned path table, surrogate ints, sorted deterministic output (encoder canonicalizes),eofcount trailer. - Two record types:
commit(hash, author ref, timestamp,isFix/isRevertsubject-regex flags — labeled heuristic) andchange(commit ref, path ref, added, deleted, rename-from). Rename chains are resolved at mine time so one path surrogate names one file lineage — unresolved renames corrupt every downstream metric. (Documented approximation: a path recreated after a deletion continues the same lineage — lineages are named by paths.) - Reports, file-level only, no model join yet:
codegraph history --report summary|hotspots|authors— churn, bus factor, bug density, momentum, firefighting frequency. - File-level city replay: buildings = files (height = running LOC sum
of numstat deltas), districts = directories, timeline scrubber in
viz, commits as ticks. (
codegraph history --serve/--city FILE: a laid-out city artifact with areplayblock — frozen union layout and keyframe height series, M9c's shape adopted early.) - Fixture: a scripted git repo built by the test suite in a temp dir
(two authors, a rename, a
fix:commit, a deletion) — deterministic, and it exercises the rename chain. - DoD: miner output byte-identical across runs on the scripted repo; summary reports match hand-counted fixture numbers; file-level replay runs end to end on codegraph's own history.
- Sampled snapshots: extract at K chosen revisions (tags/releases, or
every N commits — 50–200 frames suffice for replay) via
git worktree add, never mutating the main checkout. (codegraph snapshots [repo] --jar F (--every N | --tags) [--store S] [--src DIR]: throwaway worktree per frame,java -jarextraction,import --atappend, per-frame diagnose. RESUMABLE — revisions the store holds are skipped, so an interrupted run continues and a moved source root is handled by composing runs with different--src; a frame that fails to extract is reported and isolated, never fatal.) -
codegraph import --at <sha> [--time <t>]extendsmodel.db(M7) withrevision,entity_key(interned natural keys), andentity_version/edge_versiontables (names as TEXT — dictionary ids are file-scoped). The flat tables mirror the latest import; the cache NEVER regenerates a store holding revisions (statetemporal— it stands aside and the model is read directly). Lifespans (appeared,disappeared) are derived per key at query time — inverse indexes over the time axis, never serialized (invariant 4 applied to time). - Cross-graph queries — the ones only a tool holding BOTH graphs can
ask: hidden coupling (co-change pairs with no path — transitive,
either direction — in the declared graph) and dead weight
(declared file dependencies that never co-change), via
codegraph history --report hidden|deadweight --model M; logical coupling with support/confidence thresholds and a changeset-size cap (--report coupling,--min-support,--min-confidence). The join is by path SUFFIX (model roots sit below repo roots); ambiguous suffixes join nothing and are counted. Ownership map and truck factor shipped file-level in M9a (--report authors); code age and revisions×LOC enrichment remain open. -
codegraph timeline <id>: first/last revision containing the key, LOC series between them, still-present flag. (Exact birth commit via targeted bisect — extract one file at ~log₂ N revisions — is still open; a query-time feature, not an ingest-time cost.) - DoD: property suite (closure, profile validity, determinism) green
at every keyframe unconditionally; timeline and coupling queries
verified on a real corpus history (google/gson). ✅ Verified 2026-08-25:
all 55 gson release tags (2008–2025) in one store via two composed
snapshots --tagsruns (--src src/main/javafor the pre-2.4 layout,--src gson/src/main/javaafter the module move — resume skipping makes the composition free), zero findings at every keyframe;timeline Gsonspans 55/55 from 1.0,MappedObjectConstructordies after gson-1.7.2 (the 2.0 rewrite),Excluderis born at gson-2.1; hidden coupling finds the JsonSerializer/JsonDeserializer and Since/Until twins (no declared path), dead weight finds the stable JsonToken/TypeAdapter interfaces.
- Layout computed ONCE on the union of every key that ever existed;
every plot frozen. Buildings animate in place — rise from zero at
birth, sink at death; land is vacant before its time. Early sparse
frames are the honest picture of a city that will grow, not a
defect. (
buildEntityCity: types are buildings, modules are FLAT districts — nesting a package hierarchy from its name would be an inference; presence GAPS become explicit 0 keyframes so a rebirth scrubs honestly; members and anonymous types raise no building.) - One temporal
city.json: per building{plot, birth, death, series: [{t, height, heat, …}]}— viz interpolates between keyframes and still renders a laid-out artifact only. (codegraph replay [--store S] [--out F] [--serve]: analyzerreadEntityHistory→ citybuildEntityCity→ the M9a replay block withclock: "revisions"; metrics carried per building: loc, peak-loc, revisions, born. Verified on gson's 55 release keyframes: rev 1/55 is one sparse district of 2008, dead types leave vacant plots, JsonReader born at gson-1.6.) - Change heat as color + age as desaturation, documented like every
other channel: keyframes carry
heat(1 = changed at that tick, pre-decayed byREPLAY_HEAT_DECAYwhen a sampled revision saw the entity unchanged); the renderer keeps cooling between keyframes, ages toward gray by timeline fraction lived, and legends both — a 'Time colors' toggle (replay artifacts only) restores the plain palette. Verified on gson: 2.14.0's touched core glows ember, one-release-old changes read brick, the untouched old core grays. - Ownership as a toggleable color mode; co-change arcs visually
distinct from declared edges (they are inferences, and the city
never lies). Fed by the history join:
codegraph replay --history F(andhistory --servefor the file city) runs scm'sfileOwners+logicalCouplingand the analyzer's SUFFIX join — buildings gainowner {name, share}, the replay block gainscoChangebuilding pairs. Viz: a 'Colors' selector (Time / Owner / Plain — Owner offered only when a history was joined; unowned buildings go neutral, never a claimed hue), owner in the details panel, and co-change as DASHED magenta arcs shown for the selected building — a different KIND of line from dependency arrows, legend and help stating the inference. - DoD: replay on a real corpus reviewed as screenshots at user-facing camera angles; no per-frame allocation in the scrub path. ✅ Verified 2026-08-25 on gson (55 release keyframes + 2,088-commit history, 170 of 197 store files joined): sparse 2008 city grows to 2026; time colors read ember/brick/gray at 2.14.0 and 2.9.1; owner mode maps the original-author core against the modern maintainers; TypeAdapters shows owner (29% of added lines) and its dashed co-change fan. Scrub path allocates nothing: caller-owned Float32Array buffers, in-place instance matrix/color rewrites, event-driven repaints only.
- Per-commit incremental extraction (keyframes + deltas): Spoon's resolution is corpus-wide — the stub whitelist depends on all corpus-declared ids — so between-keyframe models are approximate. Build only if sampling proves too coarse; keyframes bound the staleness.
- Symbol rename lineage (same-parent + similar-body matching).
- Lambda/anonymous-entity tracking over time.
.mailmapauthor normalization (start with email identity; add it when it bites).
Motivation: the model says what the code is (structure) and what happened
to it (Phase 8). This phase makes it say where it lives (a permalink for
every anchor), how big and how branchy it is (measured, not guessed),
what it says verbatim (the constant values the source writes — annotation
arguments, constant initializers, defaults), and what frameworks do to it
(the DI wiring the static graph cannot see). All four land on hooks the
codebase already carries: anchors are repo-projectable the moment the header
knows the repo; the city's metric registry was written for extractor-emitted
measures (attribute:/sum: sources, metrics.ts); annotation usage is
already an edge and Spring wiring is already a documented blind spot in the
Java profile notes.
Four principles, locked up front:
- Facts in the model, projections downstream. The header gains the
repository facts (
remote,commit, repo-relativeroot); the blob-URL template of a particular host is presentation, derived by consumers — a stored URL would freeze one host's scheme into the interchange (METAMODEL §8a/§9). - Only the extractor measures.
slocandcyclomaticrequire reading source; nothing downstream may re-parse or approximate them. Gross span length stays a derived proxy (the city'sloc), and the two claims are never conflated (METAMODEL §3.8). - A value is what is written, never runtime state. A
Literal(METAMODEL §1.6) is emitted only when the language fixes it at the declaration — a literal, or an expression folding from constants; anything else isunevaluated(source text kept) or absent. Ids inside values obey closure like edge endpoints. - Framework knowledge is data, like profiles. The analyzer derives DI wiring from declared facts plus a declarative annotation table (METAMODEL §9.1); the extractor stays framework-blind, and derived wiring edges are in-memory only — invariant 4 applied to inference.
- Core: optional
repository {remote, commit, root, provider?}on the header record (Model, wire, container contract),pnpm run gen:schemascommitted with it. Additive —schemaVersionunchanged (M6 decision: the format is internal, bumping a version nobody consumes buys nothing).rootis the analyzed root relative to the repo root — the M9b gson gotcha (--src gson/src/main/java) resurfacing as a data requirement; without it no anchor projects back to a repo path. Every field is a PATTERN, not a refinement:z.toJSONSchema()drops refinements, and the published schema has to refuse an ssh remote or a../root exactly as core does, orschemas/is not the contract it claims to be. - Java extractor: passthrough flags (
--repo-remote,--repo-commit,--repo-root,--repo-provider) copied verbatim into the header — no git knowledge, no metamodel intelligence, one writer per file. The flags travel together (a remote without a sha links nowhere) and are checked against the published patterns at the flag, not written into a header the analyzer would then refuse whole. - CLI derives the values: normalize
git remote get-url origin(ssh,ssh://,git://, embedded credentials → https, strip.git);snapshotssupplies the per-frame sha it already checks out and the--srcprefix, which is already repo-relative. A remote no browser can open (local path,file://, plain http) yields NOTHING and says so — a guessed URL is a link that lies. - Analyzer/city carry
repositoryinto artifact metadata (storemetarow;readEntityHistoryexposes it beside the revisions); viz details panel renders a "view source" link from the selected entity's anchor — GitHub{remote}/blob/{commit}/{root}/{file}#L{s}-L{e}, GitLab{remote}/-/blob/{commit}/{root}/{file}#L{s}-{e}; provider guessed from hostname,providerfield overriding (self-hosted GitLab). An unknown host template renders no link. In a replay the commit is the SCRUBBED tick's sha, retargeted in place so playback churns no DOM. - DoD ✅ verified on gson at
b3f4ca2: the JsonReader link openedgson/src/main/java/com/google/gson/stream/JsonReader.java#L211-L2005on github.com at the extraction sha (screenshot reviewed); the fixture city, whose model carries no block, renders no link at all; a 4-revision gson replay store retargeted the link toed2b25d(2011) when scrubbed to revision 2/4, and that URL resolves too.
- Core:
TMetricstrait contributingmetrics: Record<string, number>(finite values validated — Zod v4'sz.number()rejects NaN/Infinity by construction; keys deliberately open — METAMODEL §3.8), optional on the Java profile's measurable kinds (types + invocables); header trait dictionary grows;gen:schemascommitted. The encoder writes the map KEY-SORTED: canonical order (MM-5) reaches inside the record, or two runs that measured the same thing would differ in bytes. - Java extractor emits
slocper type and invocable (span lines minus blank and comment-only lines) andcyclomaticper invocable: 1 + count ofCtIf,CtFor/CtForEach/CtWhile/CtDo, non-defaultCtCase(one per case expression),CtCatch,CtConditional,CtBinaryOperatorAND/OR, switch-pattern guards. A lambda's branches count toward the lambda — it is its own invocable. Purely syntactic: immune to the noClasspath resolution ceiling.slocruns a small LEXER, not a regex: a"/*"inside a string literal would otherwise open a block comment that never closes and silently blank the rest of the file. - Store: measures are ROWS (
entity_metric), not a JSON blob — the cache exists to be queried and a measure is what one aggregates; presence follows the trait set like every other trait key, sometrics: {}survives.DB_VERSION2 → 3. - City:
numericKeyreads themetricsmap (top-level loose keys stay legal but uncontractual, and the map wins), so--height sum:cyclomaticandattribute:slocwork exactly asmetrics.tspromised. - Fixture: hand-counted
cyclomaticvalues asserted on the fixture corpus (Reporting.max4 = for + if +||,join2,first2 ternary,today1); the constructs the corpus does not contain — switch labels, pattern guards, multi-catch, lambda-in-method — are hand-counted over sources written for them in the extractor'sMeasuresTest. Properties: every measure finite and non-negative,sloc ≤ span, a stub carries none. - DoD ✅ verified on apache/commons-lang at
e073d5d(15 381 entities, 5 200 measured, 9 790 total cyclomatic over 4 809 invocables):--height sum:cyclomatic --footprint locreviewed as screenshots at user-facing angles — ArrayUtils (1 099) and StringUtils (904) tower over a mid-rise of StrBuilder/Conversion/TypeUtils; the 246 unmeasured buildings (239 of them stubs) sit at the channel minimum 1, counted in diagnostics and in the legend, never at zero.sloc ≤ spanheld on all 5 200; a hand-count ofJavaVersion.get(1 + 28 case labels + 4 ifs) matched the emitted 33 exactly.
- Core:
Literalprimitive (METAMODEL §1.6) — tagged union: string, number (canonical decimal text — a Javalongdoes not fit a JSON number), boolean, null, enum (type id + simple name — a member stub is never fabricated, §6 verbatim), type, array, nested annotation, andunevaluated(source text of a constant expression the extractor did not fold).TWithValue { value: Literal }optional onattributeandmethod(annotation element defaults) in the Java profile; new edge kindannotationUse— Entity → annotation Type,arguments: {name, value}[], the implicitvalue =normalized explicit. Header dictionaries grew (one trait, one edge kind);gen:schemascommitted — the recursive union publishes as a NAMED$defs/Literal, so the contract states the tree once and both record schemas reference it. - Encoding: ids inside Literals are file-scoped surrogates, so closure
over values is enforced by the wire exactly as for edge endpoints — a
value pointing at an undeclared entity has no surrogate to name it with
and is unwritable. Property suite extended: the generators build values
from the same id pool as every other reference, so closure over values
is exercised by construction; determinism untouched — argument and array
order is written order, a source fact like parameter order. An EMPTY
argument list is omitted on the wire (the edge kind vouches for the key)
and restored on read, so
@Overridecosts no bytes. - Store:
entity.valueandedge.argumentsare JSON columns — a Literal is a tree, nothing queries inside one, and the ids it holds are the wire's own surrogates, so the blob round-trips exactly.DB_VERSION3 → 4. - Java extractor:
emitAnnotationsupgraded fromreferencetoannotationUsewith arguments (Spoon annotation values + partial evaluator; chars ride as one-character strings; a class literal in an argument still emits its ownreferenceedge — a value never replaces a dependency). Constantattributeinitializers (JLS compile-time constant expressions) and annotation elementdefaults carried viaTWithValue. A field that isfinalbut whose initializer is CODE (new StringBuilder()) carries nothing: absence means "not constant". In-place clean break for the usage edge kind — fixtures regenerated, no dual emission (the M6 precedent). - Analyzer and city: values pass through untouched on load; the details panel shows a constant beside its field, ids rendered as the referenced entity's NAME — presentation, no new derivation.
- DoD ✅ the fixture asserts
@Retention(RetentionPolicy.RUNTIME)onAuditedas anannotationUseedge whose argument is the enum form,value() default ""as aTWithValuestring,MAX_LINES = 4 * 25folded to100, and one honestunevaluated(@Audited(LedgerClient.AUDIT_TAG), a constant noClasspath cannot resolve); property suite green including value-closure; regenerated gson (63 valued entities, 599 annotation uses, 122 with arguments) and commons-lang (458 valued, 1 164 uses) diagnose clean. The discipline is visible on real code:SAFE_MAX_ARRAY_LENGTH(static final) states2147483639, whileSOFT_MAX_ARRAY_LENGTH— the sameInteger.MAX_VALUE - 8initializer, but notfinal— states nothing.
- Framework profile as data in the analyzer (METAMODEL §9.1): annotation
identity → role (
stereotype,injection-point,entry-point,qualifier, andprimary— the one role this list did not name, since@Primarynarrows by PRESENCE on the producer rather than qualifying the injection point; adding it was a row, not code, which is the property the table exists to have). Specifiable without being implemented — the profile robustness test, again: a Micronaut table validates with nothing behind it. - Analyzer DI pass: for each injection point (field / constructor param on
a stereotyped class), derive in-memory
dynamic-candidateedges from the consumer to every corpus implementation of the declared interface — candidates straight from the existinginterfaceImplementationinverse index;@Primarynarrows on presence,@Qualifieron itsannotationUseargument value (M10c) against the bean's names (its stereotype's string argument, or Spring's decapitalized default) — exact strings, not guesswork. Narrowing is never silent: the point says what it narrowed FROM. Injection points and roles are selected onannotationUseedges directly; matching reads the referenced annotation entity'sname+ its module, never a parsed id, and tolerates stub targets (the petclinic case: every Spring type is a stub).implicitSoleConstructorInjectionis profile DATA, so Spring 4.3+ constructor injection with no annotation at all is described by a field rather than by a branch. - Stereotype classification surfaced as a report (
analyze --report wiring, text and--jsonunder the same envelope) and as a semantic city color channel (codegraph city --framework spring→Building.roleplus aroleslegend block; viz gains aRolecolor mode, legended like every channel) — derivable fromannotationUseedges (M10c), no further model change. - DoD ✅ hand-verified on spring-petclinic at
a6e81a5(the last revision with@Autowired): all 6@Autowiredsites found — 9 injection points, of which the 5 injectingClinicServicelist exactlyClinicServiceImpl, the corpus's one implementation (grep -rl "implements ClinicService"returns exactly that file), and the 4 injecting Spring Data repository interfaces list NOTHING, with the reason stated: the container implements them at runtime, so the empty set is a fact about the corpus. The facts-only view is byte-identical before and after the pass —analyze --report deps --level type --declared-only --jsonis the same 107 359 bytes either side, 286 edges, provenancedeclaredonly. Also run on petclinic HEAD (818c413), which has no@Autowiredat all: 6 implicit sole-constructor injection points found by the profile's own rule.
Motivation: the second real extractor is the first test of the claim
CLAUDE.md makes on every page — that an extractor in any language can conform
using nothing but schemas/ and the container contract. C# is the right second
language: the csharp profile has existed as data since M1 without an
implementation (invariant 8 says that must be possible; M12 is where it is
checked), Roslyn gives Spoon-grade semantic binding (symbols, overload
resolution, extension-method binding), and the .NET ecosystem's legacy corpora
(Framework 4.x, WebForms, packages.config) are exactly the non-compilable
kind the pipeline exists for. The extractor must run on Linux and macOS as
first-class hosts and ship as ONE self-contained binary per OS, so a user needs
no SDK on the machine that runs it.
Three principles, locked up front:
- Roslyn without MSBuild — the noClasspath of .NET. The extractor never
opens a
.sln/.csprojand never callsMSBuildWorkspace: it walks--srcfor*.cs, parses each file, and binds oneCSharpCompilationagainst the BCL reference assemblies it carries inside itself. A missing NuGet package is a stub, not a build failure — the same degraded-honesty contract as Spoon's noClasspath (§5.2).MSBuildWorkspaceneeds an installed SDK, a restore, and an evaluable project graph: three things a legacy corpus does not offer and a self-contained binary cannot assume. A project-aware mode (--references) is an optional later enrichment, never the baseline. - The extractor holds no metamodel intelligence — the Java rule, mirrored.
It knows the C# id scheme, the kind→traits table, and how to write bytes.
Trait vocabulary, profile validation and closure live in
core; the cross-language gate (packages/core/test/fixtures-csharp.test.ts, the twin of the Java one) is where the two halves meet. - Byte-identity is the cross-OS contract. Two runs on one unchanged corpus
produce the same bytes (contract §6) — on any of the three OSes. That
single property turns "does it work on macOS/Windows?" into
cmpagainst the committed fixture snapshot, which is the only cross-OS test the Linux-only CI can delegate to a human with a laptop.
extractors/csharp/
global.json pins the SDK band (10.0.x, LTS, roll-forward latestPatch)
Directory.Build.props Deterministic, ContinuousIntegrationBuild, InvariantGlobalization,
Nullable=enable, TreatWarningsAsErrors, LangVersion latest
Codegraph.CSharp.sln
src/Codegraph.CSharp/ console project → `codegraph-csharp`
Program.cs CLI: same flags and exit codes as the Java jar (§13.5)
CorpusLoader.cs pass 0: file walk (ordinal order) + parse + one compilation
ReferenceAssemblies.cs the embedded BCL ref pack → MetadataReference.CreateFromImage
CorpusWhitelist.cs pass 1: the declared-type set (INamedTypeSymbol from source)
EntityIds.cs THE C# id scheme (§13.3)
EntityExtractor.cs pass 2
EdgeExtractor.cs pass 3
StubSynthesizer.cs pass 4
Measures.cs sloc (trivia-based) + cyclomatic (syntax-based)
Literals.cs attribute arguments / const initializers → Literal (M10c shape)
Model/ Entity, Edge, NaturalKey, JsonlWriter (Utf8JsonWriter), Progress
tests/Codegraph.CSharp.Tests/ xUnit; the same test names as extractors/java where the property
is the same (SnapshotTest, DeterminismTest, StubDisciplineTest,
ModelSchemaValidationTest, EntityTraitConformanceTest, …)
fixtures/csharp/src/ the reference corpus (§13.6) — `Acme.Order`, the Java corpus's twin
fixtures/csharp/expected/model.jsonl
Toolchain facts that bite first, mirroring extractors/java/README.md:
dotnetis not installed on the dev box (checked 2026-09-07). Install user-locally, sdkman-style:curl -sSL https://dot.net/v1/dotnet-install.sh | bash -s -- --channel 10.0 --install-dir ~/.dotnet, thenDOTNET_ROOT=~/.dotnetand~/.dotnetonPATH.scripts/lib.shgainsensure_dotnetbesideensure_jdk: PATH first, then~/.dotnet/dotnet, then adiethat prints the one-liner. Same script on macOS.DOTNET_CLI_TELEMETRY_OPTOUT=1,DOTNET_NOLOGO=1,DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1exported by the scripts so a first build is not a telemetry conversation.InvariantGlobalization=true: the extractor does no culture-sensitive work, and the flag removes thelibicuruntime dependency that makes a .NET binary fail on a minimal Linux (Couldn't find a valid ICU package) — the single most common "works on my Mac, dies in the container" failure. It also makes ordinal string behaviour identical on all three OSes, which §13.4 needs.- Roslyn packages:
Microsoft.CodeAnalysis.CSharp(syntax + semantics) only. NOTMicrosoft.CodeAnalysis.Workspaces.MSBuild, NOTMicrosoft.Build.Locator(principle 1; both are also incompatible with single-file publishing). - BCL reference assemblies: the
Microsoft.NETCore.App.Refpack'sref/net10.0/*.dllare embedded as resources at build time and loaded withMetadataReference.CreateFromImage. Why nottypeof(object).Assembly.Locationlike every Roslyn tutorial: in a single-file bundleAssembly.Locationis the empty string, so the "obvious" approach works indotnet runand silently binds nothing in the shipped binary — everystringwould become an unresolved stub and the resolution rate would collapse only in production.ReferenceAssembliesTestpins thatSystem.Stringresolves to a stub in moduleSystem, not<unresolved>, and runs against the published binary in CI (§13.7).
The profile (packages/core/src/profiles/csharp.ts) predates M6 and M10; the
first M12 commit is feat(core): csharp profile v2, a data change with the
same justification each Java change had:
| Change | Why |
|---|---|
TWithChildren added to method, constructor, lambda, delegate, property |
M6's executable-containment decision (§9.2): parameters and locals carry parent, so the container must be licensed too, or the model states a containment its own profile forbids. property because accessor bodies hold locals and lambdas. |
TMetrics optional on every type and invocable |
M10b: sloc + cyclomatic, the same two measures Java emits, so the city's --height sum:cyclomatic works on a C# corpus unchanged. |
TWithValue optional on field (const, enum members) and parameter (default values) |
M10c, the value door. An enum member is a field whose value is its constant, exactly the Java shape. |
edges annotationUse and throws added |
Attributes ARE annotation usage (M10c's edge kind, with arguments); throw statements are M10d-era evidence the insights walk consumes. |
kind event added: TNamed, TStructural, TTypedEntity, TChildOf, TSourceAnchor |
An event is a value-shaped member with its own declaration site; folding it into field would lie about the kind and into property about accessors. |
lambda disambiguator note: (file, line, column) |
The M7 lesson (§5.3): two nameless entities on one line — Chain(() => a, () => b), or a lambda inside an anonymous method — collide on (file, line). A column is a source fact. |
the dynamic note reworded: a dynamic call site is dropped and COUNTED, not emitted as dynamic-candidate |
The M10d decision moved candidate generation to the analyzer, which alone has whole-corpus implementor knowledge. An extractor inventing a candidates list from the file's usings would be the prefix-filter sin at the edge level. |
Construct → kind, on top of the profile's table:
| C# construct | kind | notes |
|---|---|---|
namespace (block or file-scoped), global namespace |
namespace |
module; global namespace is csharp:<global>; block-nested namespaces get TChildOf |
class, interface, struct, record, record struct, enum, delegate |
as named; record struct → record |
static class is a class; partial merges by id (§13.3) |
method, operator, conversion, finalizer, local function |
method |
operators use the metadata name (op_Addition, op_Implicit); a local function's parent is the enclosing invocable |
| constructor, static constructor, primary constructor (records, C# 12 classes) | constructor |
primary constructor: anchor = the parameter list; a positional record's synthesized Deconstruct/Equals are NOT emitted (never written) |
property, indexer (this[]), auto-property |
property |
indexer symbol Item(params); accessor bodies contribute edges FROM the property |
field, const, enum member |
field |
TWithValue for const and enum members |
| event | event |
+= / -= are access edges with isWrite |
parameter, local (incl. out var, pattern variables, foreach variables) |
parameter / localVariable |
var → TTypedEntity present, declaredType absent when Roslyn's inferred type is anonymous/error |
| lambda, anonymous method, local function expression | lambda |
#file:line:column |
| extension method | method + TAttachedTo → extended type |
the profile's existing rule |
| attribute usage | annotationUse edge with arguments |
attribute classes are ordinary class entities |
using X;, global using, using static X.Y, using A = X.Y |
import → module X (resp. X, the containing namespace of Y) |
folded to namespace level, never a type |
: Base, interface : IOther |
inheritance |
|
: IFoo on class/struct/record |
interfaceImplementation |
|
calls, new, delegate Invoke, nameof-free |
invocation |
virtual/interface dispatch → the declared member (profile note); dynamic receiver → dropped + counted |
| field/property/event reads and writes | access (isRead/isWrite) |
property access is modelled as access, not invocation — the source writes a member access, and the accessor is not an entity |
type usages: declared types, generic arguments, casts, is/switch patterns, typeof, default(T), base lists' generic args |
reference |
erased per §13.3 |
throw statements |
throws |
static type of the thrown expression; rethrow → the caught variable's static type |
Explicitly NOT extracted in M12, stated in the profile notes: source
generators (their output is not in --src), InternalsVisibleTo, XAML/Razor
code-behind partials (the .cs half is extracted; the generated half is
absent), #if branches the default symbol set excludes (Roslyn parses ONE
configuration — --define is a later flag), and cross-assembly DI wiring
(the analyzer's M10d framework table can gain an ASP.NET Core profile later;
the extractor stays framework-blind).
Same shape as Java's (extractors/java/.../EntityIds.java), with the
differences C# forces:
namespace csharp:<Ns.Path> csharp:Acme.Order
global ns csharp:<global>
unresolved ns csharp:<unresolved> (§13.4)
type csharp:<Ns>/<TypeMetadataName> csharp:Acme.Order/OrderService
generic type csharp:<Ns>/<Name>`<arity> csharp:Acme.Order/Repository`1
nested type csharp:<Ns>/<Outer>.<Inner> csharp:Acme.Order/OrderService.Line
method csharp:<Ns>/<Type>.<name>[`<arity>](<erasedFqnParams>) csharp:Acme.Order/OrderService.Bill(Acme.Order.Order)
constructor csharp:<Ns>/<Type>.<init>(<params>) static ctor: .<cctor>()
indexer csharp:<Ns>/<Type>.Item(<params>)
operator csharp:<Ns>/<Type>.op_Addition(<params>)
lambda / anon csharp:<Ns>/<Type>#<file>:<line>:<column>
field/prop/evt csharp:<Ns>/<Type>.<name>
parameter csharp:<Ns>/<Type>.<methodSig>#param:<name>
local csharp:<Ns>/<Type>.<methodSig>#local:<name>:<startLine>
local function csharp:<Ns>/<Type>.<methodSig>#fn:<name>(<params>)
stub type csharp:<Ns>/<TypeMetadataName> same shape as a declared type, on purpose
- Arity is part of the symbol, because C# lets
Foo,Foo<T>andFoo<T,U>coexist in one namespace. Java erases generics to the raw name; doing so here would merge three legal declarations into one entity — the M2 overload collision one level up. Roslyn'sMetadataName(Foo1`) is a source fact, not an invention, and the backtick is not a reserved id character. - Parameter types in signatures are fully-qualified metadata names WITH their
type arguments (
System.Collections.Generic.List1<Acme.Order.Order>) — **corrected in the M12c audit**: the plan first said "erased", and Humanizer overloadsHumanizeonFunc<T,string>versusFunc<T,object>, which C# allows and Java's erasure never could; erasing merged the two into one key. Arrays asT[],ref/out/indropped (they cannot overload by themselves), nullable annotations dropped (string?andstringare one type),NullableasSystem.Nullable1<T>, tuples asSystem.ValueTuplen<…>, pointers asT*, type parameters as their ordinal!0/!!0(ECMA-335's form — type-level vs method-level — computed fromITypeParameterSymbol.Ordinal; a type parameter's *name* is not part of the signature in C#, and two overloads differing only inT`'s name would otherwise get two ids). - Partial types and partial methods are ONE entity. The anchor is the
declaration that sorts first by
(file path, start line)— ordinal order, so the choice is stable across OSes — and every edge carries thesourceFileof the declaration part that produced it (the profile's own note).PartialTypesTestasserts one id, one entity, both files' edges. - Paths in anchors and
definedInuse/on every OS and are relativized against the deepest common ancestor of the--srcroots, as Java does.
Roslyn does not invent FQNs — but it has its own way of lying, and the discipline is the same: membership is the whitelist of corpus-declared type symbols built in pass 1, never a name-prefix test.
- Whitelist = every
INamedTypeSymbolwhoseDeclaringSyntaxReferencesare in the compilation's own trees (compilation.Assembly.GlobalNamespacewalked, or theTypeDeclarationSyntaxset — the test asserts both agree). - Metadata-resolved external types (the BCL from the embedded ref pack;
later,
--referencesDLLs) are stubs with their real namespace:csharp:System/String{kind: class, traits: [TNamed, TType], isStub: true},parent→ the stub namespacecsharp:System{definedIn: [], isStub: true}. The M3 stub-containment decision applies unchanged. - Error types (
IErrorTypeSymbol, a name Roslyn could not bind — a missing NuGet package, a typo, generated code) are stubs in the reserved modulecsharp:<unresolved>, named as written (csharp:<unresolved>/JsonConvert). This is the honest form: the corpus wrote a type of that name, and nothing says where it lives. Guessing a namespace from the file'susingdirectives would be inventing an FQN, which is the one thing the Java extractor exists to be defended against. Theimportedge tocsharp:Newtonsoft.Jsonfrom theusingdirective still says what was imported. - Roslyn's equivalent of Spoon's receiver promotion: an unbound member
access
foo.Bar()wherefooitself is unbound gives an error type namedfoo. It lands in<unresolved>like any other — and, like the Java note says, a stub count is not a count of external types. - Not entities: primitives are BCL types in C# (
intISSystem.Int32), so unlike Java they DO resolve — to stubs incsharp:System.void,dynamic, anonymous types, tuples' element names, pointer/function-pointer types,null/error values and type parameters are not entities;declaredTypeis omitted for them. Arrays fold to the element type ONLY for the written type reference, never for members (xs.Lengthis an access onSystem.Array'sLength— Roslyn binds it there, so the phantom the Java extractor had to fight does not arise, and the test that pins it is kept anyway). - Resolution summary on stderr, in the Java format: type references,
resolved / unresolved (
IErrorTypeSymbol), rate; entities (stubs); edges (self-edges dropped;dynamiccall sites dropped). Type parameters and error values excluded from the denominator, for the reasons §5.3 records.
codegraph snapshots orchestrates extraction with a fixed argument shape; M12
writes that shape down as the contract every extractor must honour, in
schemas/README.md §8, and the C# extractor implements it verbatim:
<extractor> [--src <dir>]… [--out <file>] [--progress auto|plain|none] [--no-progress]
[--repo-remote <url>] [--repo-commit <sha>] [--repo-root <path>] [--help]
exit codes: 0 ok · 1 failure · 2 usage · 3 unimplemented
stdout: nothing but `--help`; stderr: progress (tty only under auto) + the summary
CLI change: --jar FILE becomes --extractor FILE (--jar kept as an alias).
A .jar runs as java -jar FILE …; anything else runs directly. The seam is
already an injected Extract function in snapshots.ts, so the change is one
dispatcher plus its ENOENT messages naming the right runtime. replay/history
usage strings follow. codegraph itself stays extractor-agnostic — there is no
codegraph extract, by design: the Node side never learns a language.
Red-green, in this order, each step a commit:
-
Walking skeleton (M12a, 2026-09-07) —
header/f/eof, namespaces and every type kind (delegates with their parameters), doc comments,import+inheritance+interfaceImplementationedges, stubs; schema-validated per line (JsonSchema.Net, draft 2020-12) plus the sequence rules in the extractor's own suite (51 tests); snapshot atfixtures/csharp/expected/model.jsonl(20 files, 42 entities of which 11 stubs, 24 edges); the core gatefixtures-csharp.test.ts(13 tests: parse →validateModel(csharpProfile)zero issues → byte-identical toencodeModelToStringon the first run → closure → no self-reference → the stub evidence)../bin/codegraph validatereports OK andanalyze --report depsfolds the import layer to 10 namespaces / 6 dependencies. The JSON writer, ordering and the id scheme were proven against core's encoder before a single member existed — which was the point. -
Fixture corpus
fixtures/csharp/src(Acme.Order, the Java corpus translated, plus what C# adds): a partial class across two files; a file-scoped and a block namespace; nested types;Foo/Foo<T>in one namespace; overloads differing by generic arity; an extension method; a lambda and a local function starting on the same line;record,record struct,struct,enum : byte,delegate,event, indexer,operator +,const+ enum members, a primary constructor,using static,global using, an aliasusing; async/LINQ over the BCL (stubs with real namespaces); an unresolvable external (Newtonsoft.Json);throw, rethrow and a guard clause; attributes with positional, named andtypeofarguments; adynamiccall site (dropped + counted). -
Members and edges (M12b, 2026-09-07) — methods, constructors (the implicit parameterless one included; a record's primary constructor at its parameter list), operators, properties and indexers, fields, events, parameters with defaults, locals (
#local:name:line:column), lambdas (#file:line:column), local functions (#fn:);invocation(calls,new,: this()/: base(), user-defined operators, delegate invocations folded to the delegate type, extension calls resolved to the static method),accesswithisRead/isWrite,referencefrom the narrowest declared owner,throws(rethrow → the catch's type),annotationUsewithLiteralarguments named after the bound constructor's parameters; extensionattachedTo;sloc+cyclomatic. Members Roslyn synthesizes (recordEquals, accessors, backing fields) are not entities — a call to one folds to its type. 25 new .NET tests (MembersTest,EdgesTest,MeasuresTest); the snapshot went from 88 to 570 lines (244 entities, 285 edges) and stayed byte-identical to core's encoder. Review of the first full snapshot caught five defects before they were committed: attribute arguments leaking as accesses, a lambda parameter keyed without its lambda's position,base.Poston an unresolved base collapsing into the module key, an unbound receiver (JsonConvert) producing no edge, and aforeachlocal owning its whole loop body. -
Stub discipline tests: a corpus type declared in namespace
System.Acmestays internal (no prefix test);Newtonsoft.Json.JsonConvertlands in<unresolved>as a reference from its call site;stringlands inSystem; every nameless entity on a shared line gets its own id, and so does each of their parameters; no entity is its own parent. -
Determinism: two runs byte-identical; file walk order ordinal;
\r\nsources give the same lines as\nsources (Windows checkouts withcore.autocrlf);Utf8JsonWriterwithJavaScriptEncoder.UnsafeRelaxedJsonEscapingmatched against core'sJSON.stringifyon thefixtures/unicodecharacters — the byte-identity test in core is what catches an escaping mismatch, so the unicode identifiers go into the C# fixture too. -
CLI e2e on the C# fixture:
validateOK,analyze --report deps(55 type nodes / 114 dependencies),coupling,cycles(three module self-dependencies, no tangle),city(9 districts, 55 buildings, 98 arrows),navigator(181 nodes, 219 rows),domain-facts(26 type dossiers, 69 operations),explain --dry-run(an L8 walk plan). The fixture joined the per-fixture suites: analyzerconformance-csharp(the acceptance gate, clean), city and navigatorcsharp-fixture, and the CLIvalidatesuite. -
Real-corpus audit (M12c, 2026-09-07) — Humanizer (
src/Humanizer, the commons-lang analogue),dotnet/eShop(src, the petclinic analogue) and OrchardCore (src, the fineract analogue), extracted with the publishedlinux-x64binary, every modelvalidate-clean:| corpus | files | entities (stubs) | edges | resolution | wall clock | |---|---|---|---|---|---| | Humanizer | 212 | 12 146 (148) | 27 234 | 97.3 % | 6.2 s | | dotnet/eShop | 498 | 7 396 (789) | 13 223 | 63.5 % | 12.2 s | | OrchardCore | 5 193 | 88 007 (2 175) | 225 184 | 93.4 % | 53.5 s | **The first run aborted on all three**, each with a duplicate natural key — the audit's whole purpose. Humanizer overloads `Humanize<T>` on `Func<T,string>` versus `Func<T,object>`, which C# allows and erasure merged: signatures now carry type arguments. eShop's ten services each declare `static class Extensions` and a top-level `Program.cs`: one compilation makes them one type with ten `<Main>$`, and "first wins" would have erased nine services' DI wiring — every duplicate is now kept, re-keyed by its file. OrchardCore's `JsonDynamicValue` has 37 `explicit operator`s differing only in return type (now part of a conversion's signature) and a `(_, _) =>` lambda whose two parameters share a name (the second carries its ordinal). Humanizer then failed `validate` on an empty `name`: C# 14 `extension(T t) { … }` blocks, which Roslyn models as nameless nested types — their members are now members of the enclosing static class, attached to the receiver. **Resolution, categorised.** Before the audit OrchardCore stood at 62.7 % and its most-referenced "unresolved" names were `Task`, `Task<T>`, `IEnumerable<T>`, `List<T>`: not missing packages but the SDK's *implicit global usings*, which live in the generated `obj/` file the extractor skips as build output. Adding Microsoft.NET.Sdk's seven by default (`--implicit-usings sdk`; `web` opt-in — the Web SDK's additions made OrchardCore's own `StartupBase` ambiguous with `Microsoft.AspNetCore.Hosting.StartupBase` 345 times) and embedding the ASP.NET Core reference pack beside the BCL's took OrchardCore to 93.4 %, its edges from 178 k to 225 k, and Humanizer to 97.3 %. What is left is exactly the dependency surface §5.3 predicts: third-party packages (eShop: EF Core, Npgsql, MAUI, CommunityToolkit.Mvvm; OrchardCore: Fluid, YesSql, GraphQL, OpenIddict), generated code whose output is not under the roots (Humanizer's source generators), and one honest limit of a single compilation — two corpus types with one simple name whose projects each `global using` their own namespace (eShop's two `CatalogItem`) are ambiguous once merged. eShop's 63.5 % is that floor: the corpus mixes ASP.NET Core, MAUI and Aspire projects that never compile together. Screenshots of the eShop city and navigator reviewed at user-facing angles (§13.9). Numbers and causes recorded in the profile `notes`.
No second C# extractor exists, so the cross-validation oracle rule does not apply; the resolution-rate categorisation and the BCL-stub namespace check are the substitutes.
Yes — dedicated binaries for macOS, Linux and Windows are one dotnet publish matrix, and all of it cross-compiles from a single Linux job.
| RID | artifact | notes |
|---|---|---|
linux-x64 |
codegraph-csharp-linux-x64 |
the dev box and CI; InvariantGlobalization means no libicu needed |
linux-arm64 |
codegraph-csharp-linux-arm64 |
Graviton / Pi / Docker on Apple silicon |
osx-arm64 |
codegraph-csharp-osx-arm64 |
Apple silicon; chmod +x after download; unsigned ⇒ Gatekeeper quarantine (xattr -d com.apple.quarantine) — documented, notarization is a later decision |
osx-x64 |
codegraph-csharp-osx-x64 |
Intel Macs; also runs under Rosetta on arm64 |
win-x64 |
codegraph-csharp-win-x64.exe |
unsigned; SmartScreen warns once |
Publish shape, decided:
dotnet publish src/Codegraph.CSharp -c Release -r <rid> --self-contained \
-p:PublishSingleFile=true -p:EnableCompressionInSingleFile=true \
-p:PublishReadyToRun=true -p:IncludeNativeLibrariesForSelfExtract=true \
-p:DebugType=none -o dist/<rid>
- Single-file self-contained, not NativeAOT. Roslyn is not AOT-clean
(
Microsoft.CodeAnalysisemits trim/AOT warnings and relies on reflection for its own services), and NativeAOT needs the target OS's native toolchain, so macOS and Windows binaries would each need their own runner. Single-file bundles the runtime (~30 MB compressed) plus Roslyn and the ref pack; ~70–90 MB per RID, ~1 s cold start with ReadyToRun. Trimming stays OFF (PublishTrimmed=false) for the same Roslyn reason; revisit both when Roslyn ships AOT annotations — that is a build-flag change, not a code change. - Deterministic builds (
Deterministic,ContinuousIntegrationBuild,PathMap) so the binary, like the model, is reproducible from a commit. - Version =
extractor.versionin the header = the assembly's informational version, set from one property inDirectory.Build.props;--versionprints it;HeaderTestpins that the three agree.
Wiring into the repo's scripts and CI:
build.sh --csharp(and--allincludes it whenextractors/csharp/exists —have_csharp_extractor,ensure_dotnet); artifact report listsextractors/csharp/dist/<host-rid>/codegraph-csharp.build.sh --csharp --publish-allruns the five-RID matrix.test.sh --csharp→dotnet test, plus the published-binary smoke test: rundist/<host-rid>/codegraph-csharp --src fixtures/csharp/src --out $tmpandcmpagainst the committed snapshot. That the smoke test runs the published artifact, notdotnet run, is the point — it is the only test that sees the embedded-ref-pack path (§13.1) and the single-fileLocationtrap.- CI moved to GitHub Actions with the repository (M12c, 2026-09-07) —
.github/workflows/ci.ymlreplaces.gitlab-ci.yml, and GitHub's hosted macOS and Windows runners turn the cross-OS check from a laptop ritual into a gate:verify(the TypeScript workspace: build, typecheck, tests,schemas/drift) andjava(./mvnw test) — the old gate, ported;csharp-test—dotnet teston the SDK pinned byglobal.json;csharp-publish— a five-job matrix, one RID each, every job calling./build.sh --csharp --rid <rid>so CI and a developer's machine share one definition of the publish flags; each binary is uploaded as an artifact (14 days);csharp-smoke— a second matrix downloading each artifact onto a runner of ITS OWN OS (ubuntu-latest,ubuntu-24.04-arm,macos-latestfor Apple silicon,macos-15-intel,windows-latest), running it onfixtures/csharp/srcfrom the repo root with the relative--src, andcmp-ing the output with the committed snapshot. Nothing is installed on those runners for the extractor: the binary carries the runtime. A.gitattributes(* text=auto eol=lf,*.jsonl -text) pluscore.autocrlf falsekeep the snapshot's bytes intact on Windows;release— on av*tag, the five binaries renamed after their RID plus aSHA256SUMSare attached to a GitHub Release. Measured locally first: the five-RID matrix cross-publishes from one Linux host in 287 s (57 s per RID, ReadyToRun included), andfileconfirms each artifact is a native executable of its target (ELF x86-64/aarch64, Mach-O x86_64/arm64, PE32+).
README.md+CLAUDE.mdarchitecture block gainextractors/csharp/;extractors/csharp/README.mdmirrors the Java one (build, run, progress, the stderr summary, the toolchain gotchas above).
| Hazard | Where it shows | Guard |
|---|---|---|
Assembly.Location == "" in single-file bundles |
published binary binds no BCL; dotnet run is fine |
embedded ref pack + smoke test on the published artifact |
libicu missing |
Linux containers, Alpine | InvariantGlobalization=true; the CI image test |
| Case-insensitive file systems | macOS, Windows: Foo.cs and foo.cs cannot coexist, and enumeration order differs |
ordinal sort of the walked paths before parsing; DeterminismTest |
\ path separators |
Windows anchors and definedIn |
always emit /; PathsTest on a \-joined input |
\r\n sources |
Windows checkouts | Roslyn line map handles it; sloc scanner test with both endings |
Culture-sensitive string.Compare/ToUpper |
wrong canonical order on a Turkish locale | StringComparer.Ordinal everywhere; an analyzer rule (CA1309) as an error |
Utf8JsonWriter escaping <, >, &, non-ASCII by default |
<global>, <init>, unicode names |
UnsafeRelaxedJsonEscaping + the core byte-identity gate |
| Gatekeeper quarantine | first run on macOS after a download | documented; not a code problem |
| Long paths | Windows MAX_PATH on deep corpora |
\\?\ not needed on .NET 10 with long-path awareness; --src a deep fixture in the Windows manual check |
- M12a — skeleton + contract ✅ (2026-09-07): profile v2 (core), toolchain
scripts (
ensure_dotnet,build.sh --csharp [--publish-all],test.sh --csharpwith the published-binarycmp), walking skeleton end to end (§13.6 first bullet),snapshots --extractor(a.jarruns underjava -jar, anything else directly;--jarkept as an alias), and the extractor command-line contract asschemas/README.md §8, generated from core. - M12b — the model ✅ (2026-09-07): members, all edge kinds, stubs, measures, literals, the full fixture and its snapshot, stub-discipline and determinism tests, CLI e2e, and the fixture in every per-fixture suite.
- M12c — distribution + audit ✅ (2026-09-07): five-RID publish matrix
(cross-published from one Linux host in 287 s, each artifact a native
executable of its target), GitHub Actions gate with the per-OS smoke test
and tagged releases, the three-corpus audit (five defects fixed, resolution
causes measured, implicit usings and the ASP.NET Core pack added), eShop
city (187 districts, 1 176 buildings, 3 738 arrows) and navigator (4 493
nodes, 10 751 rows, 7 cycles) reviewed as screenshots, profile
notesrewritten from measurements,docs/csharp-extractor.mdfor running it with the SDK, the runtime, or nothing.
Definition of done: fixtures/csharp/expected/model.jsonl byte-identical to
core's encoder and profile-valid with zero issues; the same bytes from the
published linux-x64, osx-arm64 and win-x64 binaries on their hosts; the
audit numbers in the profile notes; ./test.sh and CI green with the C#
fixture in every per-fixture suite.
The Spoon extractor ships the same way: one executable per platform, no runtime to install. The mechanism is not the same, and the difference is not cosmetic.
native-image cannot cross-compile. dotnet publish -r osx-arm64 works from
Linux because .NET ships prebuilt per-RID runtimes and the app stays portable IL.
GraalVM instead runs an AOT compiler and the host's native linker, so a macOS
binary is built on macOS. The C# matrix is five RIDs in one Linux job plus five
smoke jobs; the Java matrix is five jobs, each building and proving its own
RID (java-native in .github/workflows/ci.yml).
| RID | runner | artifact |
|---|---|---|
linux-x64 |
ubuntu-latest |
codegraph-java-linux-x64 |
linux-arm64 |
ubuntu-24.04-arm |
codegraph-java-linux-arm64 |
osx-arm64 |
macos-latest |
codegraph-java-osx-arm64 |
osx-x64 |
macos-15-intel |
codegraph-java-osx-x64 |
win-x64 |
windows-latest |
codegraph-java-win-x64.exe |
Build shape, decided:
native-image -jar target/codegraph-java.jar -o dist/<rid>/codegraph-java \
--no-fallback -march=compatibility
- From the shaded jar, deliberately: the binary and
java -jarare then the same artifact compiled two ways, and cannot drift. Reachability metadata and the platform reference travel inside that jar. --no-fallback— a missing reflection entry must fail the BUILD, never produce an image that silently needs a JVM.-march=compatibility— the baseline ISA, not the builder's CPU. A released binary runs on the machine that downloads it.- ~42 MB, ~3 ms startup, and 13× faster than the jar on the fixture corpus (0.13 s vs 1.78 s) — no JIT warm-up on a process that exits in a second.
The hard part was not the compiler: it was the platform library. ECJ, given
no -bootclasspath, resolves java.* against the class library of the JVM it
runs inside. A native image has no JVM, no java.home, and no boot classpath —
and Spoon's noClasspath mode does not degrade gracefully when java.lang is
absent, it INVENTS: System.out.println(…) yields a type named out, a type
variable T becomes an entity. Measured on fixtures/java/src: 84.9%
resolution and 40 stubs, against 94.5% and 27.
So the binary carries its own, exactly as the C# extractor embeds the BCL
reference pack (§13.1). Two build outputs, both generated from the build JDK's
lib/ct.sym by PlatformReferenceJar and both landing inside the shaded jar:
java-api.jar— the Java 17 public API as signature-only class files, 4,721 classes / 2.6 MB (a JDK image is ~130 MB). Handed to Spoon as the source classpath, which ECJ searches LAST — bootclasspath, extdirs, sourcepath, classpath. On a JVM the VM's own library still wins, sojava -jarbehaviour is untouched and the committed snapshot did not move.- A reachability-metadata file naming
java.base's 1,352 types. Spoon resolves an import through the runtime classloader, not through ECJ — proved by running the jar under--limit-modules java.base, which flips exactly thejava.sql.*imports fromdeclaredtoderivedwhile ECJ still sees them.java.baseonly: registering the rest pulls AWT, fontmanager and sound natives into the image and the single file becomes a directory of shared objects.
Two smaller obstacles, both recorded because they are invisible until they bite:
ECJ's handleExtdirs dereferences getJavaHome() unconditionally, so the image
NPEs before parsing a single file unless java.ext.dirs is set (empty is the
truth — extension directories are a JDK 8 relic); and the reflection metadata for
Spoon/ECJ itself is agent-traced over three corpora and committed under
src/main/resources/META-INF/native-image/, because a trace of one corpus misses
what another needs.
The residual, stated exactly. On fixtures/java/src the binary reproduces
the committed snapshot byte for byte. On gson (86 files, 3,635 entities, 9,260
edges) the entity table is byte-identical to the jar's and 3 of 9,260 edges
differ: imports of java.sql.* carry derived instead of declared. The
package they point at is unchanged, no entity is affected, and it is the same
answer a JVM limited to java.base gives. That is provenance doing its job —
the metamodel has a word for "inferred, not read" precisely so this can be
reported rather than hidden.
Wiring, mirroring the C# story so the two cannot disagree:
./build.sh --java --native→extractors/java/dist/<host-rid>/codegraph-java;ensure_native_imagefinds GraalVM (GRAALVM_HOME, JAVA_HOME, PATH, sdkman)../test.sh --javaruns./mvnw testand then, if a binary exists, the native smoke test: the binary re-extracts the fixture corpus and must reproduce the snapshot. It is the only test that sees the image's platform-library path, and--no-fallbackcannot catch a wrong one.- CI builds and smoke-tests all five RIDs and attaches them to a
v*release beside the C# binaries.
Motivation: the third real extractor closes the loop the plan opened in §10 —
run codegraph on codegraph. It is also the first extractor written in the
same language as core, which makes it the sharpest test of the boundary rule:
when importing @codegraph/core is one line away, an extractor must still
conform using nothing but schemas/. TypeScript brings the two metamodel
features no other profile exercises — Entity.space (type vs value) and
declaration merging — and its own flavour of non-compilable legacy: the
pre-ES-module codebase of namespace blocks and /// <reference path>
directives compiled with outFile, which no modern toolchain builds any more.
What is the Spoon / Roslyn of TypeScript? The answer is the language's own
compiler, used as a library — the typescript npm package, the same package
that ships tsc:
| Roslyn | TypeScript compiler API | Role |
|---|---|---|
CSharpCompilation |
ts.Program (ts.createProgram) |
one compilation over a set of files and options |
SyntaxTree / SyntaxNode |
ts.SourceFile / ts.Node |
the concrete syntax tree, positions included |
SemanticModel |
ts.TypeChecker (program.getTypeChecker()) |
binding and typing on demand: getSymbolAtLocation, getTypeAtLocation, getResolvedSignature |
ISymbol |
ts.Symbol — with declarations[], plural |
the binder's view; the plural is declaration merging made explicit |
MetadataReference (the BCL ref pack) |
lib.*.d.ts inside the package |
the standard library, resolved from the package's own directory |
IErrorTypeSymbol |
the any/error type from an unresolved name |
binding failure, tolerated, never fatal |
It is error-tolerant by construction: a Program over files with missing
modules, missing types and type errors still binds everything that binds —
tsc prints diagnostics and emits anyway. So "noClasspath" is not a mode to
switch on, it is the default; the extractor's job is to keep the honest
distinction between "resolved to a declaration" and "resolved to any".
The alternatives, and why they are not the baseline:
| Candidate | What it is | Verdict |
|---|---|---|
| ts-morph | a convenience wrapper over the compiler API (Spoon-like navigation, findReferences) |
rejected — wraps every node in an object, so memory and time scale badly on the corpora this pipeline exists for; lags TypeScript releases; adds nothing the extractor needs beyond the checker it already wraps |
| tree-sitter, swc, oxc, Babel, esbuild | parsers — fast, error-tolerant, no binder | rejected as a baseline — without a checker every call is a dynamic-candidate and every import an unresolved string; the model would be a lexical approximation of the graph. A syntactic pass is not needed for speed either: the compiler API parses a million lines in seconds; the checker is what costs |
| scip-typescript (Sourcegraph) | an indexer built on the same compiler API, emitting SCIP occurrences | not for this extractor — SCIP carries definition/reference/implementation occurrences, not kinds, traits, provenance, access vs invocation, or spaces. It remains the cheap multi-language adapter path §10 lists, whose output is a lower bound of this one |
ts.createLanguageService |
the editor-facing incremental layer | unnecessary — one batch program per run; incremental extraction is the temporal track's problem (§11.4) |
Three principles, locked up front:
- The checker without a build — the noClasspath of TypeScript. The
extractor never runs
tsc, never builds project references, and never needsnode_modules. It walks--srcfor*.ts,*.tsx,*.mts,*.cts(and*.js/*.jsxunder--allow-js), skippingnode_modulesand build output, and creates ONEProgramover all of them. Atsconfig.jsonis read for resolution options only —paths,baseUrl,rootDirs,jsx,lib,target— never forfiles/include(the roots define the corpus) and never forreferences. A module specifier that resolves to nothing is a stub module keyed by the specifier, not a dropped edge; a package that is not installed is a stub, not a failure. One resolution the standard host does not do, and this one does: a bare specifier that names a package declared inside the roots (@codegraph/core→ thepackage.jsonunderpackages/core) resolves to that package's source entry even with nothing installed and nothing built — the monorepo's own packages are corpus, not dependencies. - No metamodel intelligence — and no
@codegraph/*at runtime. The extractor's only runtime dependency istypescript. A boundary test (the twin ofpackages/llm/test/boundary.test.ts) pins that nothing underextractors/typescript/srcimports@codegraph/; core is a devDependency for the extractor's own tests, where validating one's output against the reference implementation is exactly right. Two reasons: the core-side gate — "byte-identical to what core's encoder writes" — is a statement about two independent encoders only if they are independent; and a package that depends ontypescriptalone can be published and run withnpxon a machine that has never seen this repo. - Byte-identity across OS and across install state. Two runs over one
unchanged corpus write the same bytes on Linux, macOS and Windows — the C#
contract — and, because external entities are keyed by package name
(from the nearest
package.json) and never by a resolvednode_modules/…/index.d.tspath, the keys of a model do not depend on what happens to be installed. Edge counts do (an uninstalled package's exports areany, and a call throughanyis dropped and counted), and the fixture snapshot is produced with nonode_modulesat all, so the committed bytes are reproducible from a bare checkout.
extractors/typescript/ pnpm workspace package `codegraph-typescript` (public name; bin of the same name)
package.json dependencies: typescript (catalog) — NOTHING else at runtime
devDependencies: @codegraph/core, vitest, fast-check, tsup
tsconfig.json strict, NodeNext, extends ../../tsconfig.base.json
src/
main.ts CLI: the extractor contract (schemas/README.md §8) + --tsconfig, --allow-js, --ignore-node-modules
corpus.ts pass 0: walk (ordinal order), tsconfig resolution options, the workspace-package resolver, ts.createProgram
whitelist.ts pass 1: the declared-symbol set (every declaration under the roots and outside node_modules)
ids.ts THE TypeScript id scheme (§14.3): keys, escaping, spaces
entities.ts pass 2
edges.ts pass 3
stubs.ts pass 4: lib / package / <unresolved> stubs, stub modules for specifiers
measures.ts sloc (ts.createScanner — the compiler's own lexer, JSX and template literals included) + cyclomatic
literals.ts const initializers, enum values (checker.getConstantValue), decorator arguments, parameter defaults
model/ NaturalKey, Entity, Edge, canonical order, JsonlWriter (JSON.stringify — the reference bytes, natively), Progress
test/ vitest; the C# suite's names where the property is the same (snapshot, determinism, stub-discipline,
ids, schema-per-line, cli, boundary) + spaces, merging, escaping
bin/codegraph-typescript `#!/usr/bin/env node` → dist/main.js
fixtures/typescript/src/ the reference corpus (§14.6): `acme-order`, the Java and C# corpora's twin, plus a legacy script half
fixtures/typescript/expected/model.jsonl
pnpm-workspace.yamlgainsextractors/typescript;pnpm -r build/test/typecheckcover it with no new tooling.build.sh --tsalready builds it;test.sh --tsgains the built-bundle smoke test: runbin/codegraph-typescriptonfixtures/typescript/srcandcmpwith the snapshot — the one test that sees the bundled code path rather than vitest's source aliasing.typescriptstays external to the tsup bundle.ts.createProgramlocateslib.*.d.tsrelative totypescript.js(ts.getDefaultLibFilePath); a bundle that inlines the compiler binds no standard library and everyArray,Promiseandstringmethod becomes<unresolved>— theAssembly.Locationtrap of §13.1 in its Node form. A test pins thatArrayresolves to a stub in module<lib>, and runs against the built bin, not the sources.- The pinned compiler version is the workspace catalog's (
typescript^5.9); the header'sextractor.versionis the package version and the compiler version rides inextractoras an extra key (the header schema allows it), so a model says which checker produced it. codegraph snapshots --extractorlearns one more launcher rule beside.jar → java -jar: a.js/.mjs/.cjsfile runs undernode. Anything else,bin/codegraph-typescriptincluded, still runs directly.- Node ≥ 22 is already a requirement of codegraph itself, so "nothing to install" holds for anyone who can run the analyzer. A Node single-executable build is deferred (§14.7).
The profile (packages/core/src/profiles/typescript.ts) predates M6, M10 and
M12; the first M13 commit is feat(core): typescript profile v2, data changes
with the M12 pattern of justification:
| Change | Why |
|---|---|
kind constructor added: TInvocable, TWithChildren, TWithParameters, TWithLocalVariables, TWithInvocations, TWithAccesses, TChildOf, TSourceAnchor (+ TComment, TMetrics); no TNamed, no TTypedEntity |
the Java/C# shape; a class constructor is not a method named constructor — it has no return type and its parameter properties declare fields |
TMetrics optional on every type kind and every invocable; module too |
M10b: sloc + cyclomatic, so --height sum:cyclomatic works on a TypeScript city unchanged. A module's top level is executable code, hence measured |
TWithValue optional on variable (const initializers), property (enum members, readonly literals), parameter (defaults) |
M10c, the value door; an enum member is a property whose value is its constant |
TWithInvocations, TWithAccesses, TWithLocalVariables optional on module |
a module body IS a scope with statements: app.listen() at top level is an invocation FROM the module, and const x = … at top level is a local of it. Without the licence the model states edges from an entity its profile says cannot have them |
edges annotationUse and throws added |
decorators ARE annotation usage (M10c's edge kind, with arguments); throw sites are the M10d-era evidence the insights walk consumes |
arrowFunction / nameless function disambiguator: line:column |
the M7 lesson (§5.3): two arrows on one line collide on (file, line). The module is the file already, so the disambiguator needs no file component |
| merging note rewritten: a merged symbol is one entity per declaration file, one entity per file for same-file merges | the module is the file and containment is where a thing is written (invariant 5). The old note ("one namespace id may span files") contradicted both. An edge to a merged symbol targets the declaration that owns the referenced member (§14.3) |
new note: key escaping (/, #, % and, in names, .) |
the file path is the module and / is a reserved separator (§14.3) |
space on enum becomes per-entity: ["type"] for const enum |
the profile already says const enum members leave no runtime entity; the entity should say so where analyses read it |
any-receiver note reworded: dropped and COUNTED, never emitted as dynamic-candidate |
the M10d/M12 decision: candidate generation is the analyzer's, which alone has whole-corpus knowledge |
Construct → kind, on top of the profile's table:
| TypeScript construct | kind | notes |
|---|---|---|
| a source file, module or script | module |
definedIn = the one file; space: ["value"]. A script (no import/export) puts its declarations in the global scope semantically, but they are still children of the file (invariant 5); nothing lives in a <global> module |
namespace X / module X {} (internal module), nested X.Y |
namespace |
child of the file or the enclosing namespace; space from what it declares (["type"] if it exports only types) |
declare module "pkg" {} (ambient external module) |
module |
keyed by the quoted name (ts:pkg), definedIn = the .d.ts, isStub: false — it IS declared by the corpus; import "pkg" resolves to it |
declare global {} and module augmentations declare module "express" { interface Request { user: User } } |
no entity for the block | its members are ordinary declared entities parented by the augmented type — a stub type when the package is external (§14.4) |
class, abstract class, class expression bound to a name (const A = class {}, export default class {}) |
class / abstractClass |
name from the binding (default for a nameless default export); a class expression with no binding is not an entity (dropped and counted), its members neither |
interface |
interface |
extends → inheritance (multiple); space: ["type"] |
type T = … |
typeAlias |
constituents → reference edges |
enum, const enum |
enum |
members → property children with TWithValue (checker.getConstantValue); const enum → space: ["type"] |
| function declaration, overload signatures + implementation | function |
ONE entity, anchored at the implementation (or the sole ambient signature); overload signatures are not entities. signature is the implementation's |
| function expression, arrow function | function (no TNamed) / arrowFunction |
#line:column; const f = () => {} yields the variable f AND its child arrow (profile rule); call sites target the arrow |
method, get/set accessor, static member, abstract method, object-literal method |
method |
accessors: #get / #set; static: #static (§14.3) |
constructor(…) |
constructor |
parameter properties (constructor(private x: T)) yield BOTH a parameter and a property |
property declaration, accessor field, object-literal property, enum member |
property |
index signatures are not entities |
#private member |
method / property |
name escaped (%23secret), never the unescaped # |
var/let/const at any level, every binding of a destructuring pattern |
variable |
parent = module / invocable / block-owning invocable; TWithLocalVariables on the owner lists them |
parameter, this parameter excluded |
parameter |
#param:name below the owner, below the owner's own disambiguator when it has one |
| type parameter | not an entity | its constraint and default → reference edges from the declaring entity |
decorator @Dec(args) |
annotationUse edge with arguments (literals) |
legacy (experimentalDecorators) and TC39 decorators alike — both parse; the decorator is an ordinary function/variable entity |
import x from "m", import { a as b }, import * as ns, import type, export … from, export * from, import("m") / require("m") with a literal, /// <reference path="…"> |
import → the module |
module-level, never a member (the Java fold). import type is an ordinary import edge whose erasure shows in the target's space. A computed specifier is dropped and counted |
class A extends B |
inheritance |
extends Mixin(Base): the checker's base type if it names a declaration, else a reference to what is named, counted |
class A implements I |
interfaceImplementation, provenance declared |
structural conformance is NOT computed by the extractor — derived conformance is an analyzer enrichment, never mixed in |
call, new, tagged template, optional call f?.(), JSX element <Comp/> with a component tag |
invocation |
target = the resolved signature's declaration, else every declaration of the callee's symbol (a union receiver's synthetic symbol has the real ones); receiver typed any/unknown/error → dropped and counted. A JSX element is a call by the language definition, so the edge is declared; intrinsic tags (<div>) yield nothing |
property read / write / compound / delete, destructuring reads, obj["lit"] |
access (isRead / isWrite) |
computed non-literal keys dropped and counted; a property access that runs an accessor is access, not invocation (the C# rule) |
type positions: annotations, generic arguments, as, satisfies, typeof X in a type, keyof T, import("m").T, conditional/mapped-type operands, heritage clause type arguments, instanceof X |
reference |
to named types only; primitives, literal types, type parameters and any name nothing |
throw expr |
throws |
the static type of expr when it names a declaration; rethrow → the caught binding's static type; else dropped and counted |
| JSDoc / leading doc comment | comments |
the declaration's own JSDoc, not every leading comment |
Explicitly NOT extracted in M13, stated in the profile notes: structural
(derived) interface conformance; emitDecoratorMetadata synthesis; CommonJS
module.exports reassignment shapes beyond a literal require; monkey
patching (the JS profile's rules apply verbatim); Vue/Svelte/Angular template
files (only .ts/.tsx bodies are read — a .vue SFC's script block is a
later --extract-sfc flag, never a default); declare d code whose
implementation is in a language the pipeline does not model (native
addons).
module (file) ts:src%2Facme%2Forder.ts path relative to root, escaped
ambient module ts:express declare module "express" in the corpus
external package ts:zod/ZodType ts:@types%2Fnode/Buffer nearest package.json `name` under node_modules
external subpath ts:zod%2Fv4 an import edge's target: the specifier as written
lib ts:<lib>/Array lib.*.d.ts inside the typescript package
unresolved name ts:<unresolved>/Ledger a name that binds to nothing, as written
unresolved specifier ts:.%2Fmissing ts:@megacorp%2Fledger a stub module for an import nothing resolves
type / value ts:src%2Facme%2Forder.ts/OrderService nested namespaces: Ns.Inner.Type
member ts:src%2Facme%2Forder.ts/OrderService.bill no parameter list — TypeScript overloads are one declaration
static / accessors …/OrderService.create#static …/OrderService.total#get …/OrderService.total#set
constructor …/OrderService.constructor
nameless invocable …/OrderService.bill#12:7 top level: ts:src%2Fa.ts#3:15
parameter / local …/OrderService.bill#param:order …/OrderService.bill#12:7#param:x
- Escaping, the one rule.
renderIdreserves/and#in a module and#in a symbol so rendering stays injective; a file path is made of/. Every path segment and every non-identifier name entering a key is percent-encoded for exactly/→%2F,#→%23,%→%25, and — for names only, where.is the nesting separator —.→%2E. Identifiers cannot contain any of the four, so an identifier is written as-is and the encoding is injective and reversible; the extractor'sescape/unescapepair is pinned by a fast-check round-trip property. Rejected alternatives: relaxing core's reservation (breaks rendering injectivity for every language, since a module entity renders with no/after it); a look-alike separator such as U+2215 (not typeable on a CLI or in a SQL query); dropping the extension or dotting the path (a.b/c.tsanda/b.c.tscollide). Thenameof the module entity is the unescaped path, which is what the city and navigator display. - Module = file, always. The
<global>module of C# has no counterpart: a script file's globals are children of the file (invariant 5), and an entity's module is the file it is written in. Consequence for declaration merging: aninterface Orderin two files is two entities; two in one file is one entity anchored at the first. A reference to a merged symbol targets the declaration that owns the referenced member (checker.getSymbolAtLocationon the member gives one declaration); a reference to the merged container itself targets its first declaration in canonical file order — a deterministic choice, named in the profile note as one. - Overloads are one entity — TypeScript has no overloading by parameter type at the declaration level; the overload list is one function's signature set. No parameter-list component, unlike Java and C#.
- Same-name members that TypeScript allows to coexist get a
disambiguator:
staticfor static members (a class may declarestatic parse()andparse()),get/setfor accessor pairs. The disambiguator is the only place these flags are identity, so a class with only an instanceparserenders with no suffix — the absent-first canonical order puts the instance member first. - Nameless invocables carry
line:columnof their first token, below the nearest named ancestor's symbol and below that ancestor's own disambiguator when it has one — so two arrows on one line are two entities, and an arrow inside an arrow does not become its own parent (the METAMODEL §1.1 column rule). At module top level the symbol is empty and the disambiguator alone identifies the entity:ts:src%2Fa.ts#3:15. METAMODEL §1.1 says the symbol is "empty only for a module"; M13a loosens that by one clause ("or for a nameless entity at module top level, which then carries a disambiguator"). Verified (2026-09-08): the JSONL reader recognises a module record bym === i, andrenderIdonly omits the/for an empty symbol — nothing incoreor the analyzer treats an empty symbol as the module marker, so the form needs no new encoding. - Stub keys have the same shape as declared keys — membership is the
whitelist, never the key. An external type's module is the npm package
name (walk up from the declaring file to the nearest
package.json), never the resolved path undernode_modules, so keys are the same on every machine; an import edge's stub module is the specifier as written (zodandzod/v4are two entry points, honestly two modules). Node built-ins are normalised to thenode:form (fsandnode:fsare one module in fact, andmodule.builtinModulessays which names those are).
Pass 1 builds the whitelist: every ts.Symbol at least one of whose
declarations lies in a file under the roots and outside node_modules. Then:
| The checker says | The model says |
|---|---|
| declared in the corpus | a declared entity |
declared only under node_modules/<pkg> |
stub type in module <pkg>; a member reference into it retargets to the type stub (the C# fold); a free function or variable exported by the package retargets to the stub module — the smallest degraded container a module-level value has |
declared in lib.*.d.ts |
stub type in <lib>; string/number/boolean/symbol/bigint/void/null/undefined and literal types are not entities (the Java primitive rule) |
an error type / cannot find name |
stub in <unresolved>, named as written — never a guess from the file's imports |
| a specifier that resolves to nothing | a stub module keyed by the specifier; the import edge survives (import fan-out stays honest) |
any / unknown / an index signature on a receiver |
the edge is dropped and counted; the count is the profile's stated resolution ceiling |
| a corpus declaration merged into an external one (module augmentation) | the corpus members are declared entities parented by the stub type — closure holds because the stub exists; isStub is a fact about the type, not about its members |
the typescript package itself, when a corpus imports it |
an external package like any other (ts:typescript/Node) — the extractor's own dependency is not corpus |
Two things the checker resolves that are still external: the standard library
(<lib>) and installed packages. Resolvability is not membership — the C#
rule, verbatim.
The workspace-package resolver (principle 1) is the one place the
extractor adds resolution the compiler host lacks, and it is bounded: a bare
specifier whose package name matches a package.json name under the roots
resolves to that package's source entry — the source/module/main/types
field that points at a file under the roots, else src/index.ts when it
exists — and only when standard resolution failed. It never reads
node_modules to do so, never follows exports conditions into dist/
(build output, not corpus), and names every such resolution on stderr under
its own counter, because it is a fact about the repository layout rather than
about the language. A subpath into a workspace package (@acme/pricing/rules)
resolves the same way below that package's source root, else stubs.
Same flag shape and exit codes as the Java jar and the C# binary
(schemas/README.md §8) — codegraph snapshots must stay language-blind.
Three additions, all optional:
| Flag | Meaning |
|---|---|
--tsconfig <file> |
the tsconfig.json whose resolution options apply (paths, baseUrl, rootDirs, jsx, lib, target, allowJs); default: the nearest tsconfig.json at or above each --src root, the first root's winning on conflict (named on stderr); none for the synthesized defaults (target esnext, moduleResolution bundler, lib esnext + dom, jsx preserve, skipLibCheck, noEmit) |
--allow-js |
also walk *.js/*.jsx/*.mjs/*.cjs; JSDoc types feed declaredType (the JS profile's rule); the model still claims lang: "ts" — a mixed corpus is a TypeScript program with JavaScript files in it, which is what the checker models too |
--ignore-node-modules |
never read node_modules, even when present: every external package is an unresolved stub module. What the fixture uses, and what makes two machines' models key-and-edge identical |
stdout carries nothing but --help/--version; progress and the resolution
summary go to stderr:
✓ walk 310 files 0.1s
✓ program 310 files, tsconfig ./tsconfig.base.json, lib esnext 2.4s
✓ whitelist 4 812 symbols 0.3s
✓ entities 9 340 entities 1.1s
✓ edges 21 907 edges 1.8s
✓ stubs 417 stubs (lib 96, packages 288, <unresolved> 33) 0.1s
✓ write 31 664 records 0.4s
RESOLUTION SUMMARY
type references : 21 907
resolved : 20 588
unresolved : 1 319
resolution rate : 94.0%
any-typed receivers (dropped) : 212 ← the honest ceiling (profile note)
imports : 1 204 (unresolved: 41, workspace-resolved: 388)
entities : 9 757 (stubs: 417)
edges : 21 907 (self-edges dropped: 3, computed accesses dropped: 19, nameless classes dropped: 1)
wrote model.jsonl
-
Walking skeleton first: modules and imports (resolved, workspace-resolved, unresolved → stub modules), every type kind with
space, inheritance / implements, stubs — one file in,codegraph validategreen, snapshot committed, the core gatepackages/core/test/fixtures-typescript.test.ts(parses as aModel; byte-identical to core's encoder; ZERO profile issues; closed; no self-edges; the stub-discipline evidence by id). -
The fixture corpus
fixtures/typescript/src—acme-orderas an ES module package tree plus a legacy script half — around 18 files, 350 lines, each pinning one hazard (its README lists them, the Java README's form):tsconfig.jsonwithpaths(@acme/*) and a second workspace packagepackages/pricingimported by name with nonode_modulesand nodist— the workspace resolver's case;@megacorp/ledgerimported and never present — the stub-module case;Ledgerextended from it — inheritance from a stub; a call through itsanyexport — the dropped-and-counted case;import typeand an inlinetypespecifier — erased imports whose edge is the same kind;interface Ordermerged in one file and again across two files; anamespace Acme.Orderin two script files joined by/// <reference path>— the legacy internal-module style; a module augmentation of the absent package;const enum Channeland a plainenumwith computed and literal members —space: ["type"]on one andTWithValueon both;- two arrow functions on one line, an arrow inside an arrow, a top-level
arrow (
ts:…#line:col), an IIFE; - overloads with one implementation;
static parse()besideparse(); aget/setpair; a#privatefield; a string-literal member"a.b"and a computed[Symbol.iterator]— every escaping rule; - a
.tsxcomponent tree (<OrderTable rows={…}/>) — JSX invocations; - decorators with literal arguments (both syntaxes, two files);
throw new OrderError(...), a rethrow, athrow "string"(dropped);- parameter properties, destructured parameters and locals, a
declare module "legacy-lib"ambient in a corpus.d.ts,export default class {},export * from, a dynamicimport("./lazy")with a literal and one with a template (dropped and counted), a compound assignment (isReadandisWriteboth).
-
The extractor's own suite (vitest): per-line schema validation against
schemas/*.schema.jsonand the sequence rules; the trait-key rule of contract §4; snapshot byte-identity; determinism (two runs, shuffled walk order, CRLF-converted sources,--srcgiven asa bandb a); stub discipline by id; the escaping round-trip property; the spaces of every fixture kind; merging (which declaration an edge lands on); the built bin's CLI (exit codes 0/1/2, silent stdout,--version); the boundary test;Arrayresolving to<lib>from the built bundle. -
The fixture in every per-fixture suite: analyzer (
fixture.tsloaders, conformance, coupling, cycles, store import/fold/diagnose), city, navigator, CLI e2e (validate,analyze,export,import,city,navigator,domain-facts,explain --dry-run). -
Self-hosting — the DoD of the phase.
codegraph-typescript --src packages --src extractors/typescript --out codegraph.jsonlon this repository:validateclean, zero<unresolved>entities that name a corpus symbol (every one must be a genuinely absent dependency, named in the audit),analyze --report cyclesagreeing with the package boundaries CLAUDE.md states (no cycle crosses a package;vizandnavigator-uiimport their model packages for types only — the boundary tests restated as a graph query), and the city and navigator of codegraph reviewed as screenshots at user-facing angles. -
Three-corpus audit, the M12c pattern, with resolution causes measured and the residue named in the profile
notes:- TypeScript 4.9's own
src/compiler— the legacy corpus:namespace ts {}in a hundred files merged by/// <reference>,outFile-style, no ES modules. Merging, script-file globals and the per-file entity rule at scale; - nestjs/nest — decorators everywhere, a pnpm monorepo of packages
importing each other by name: the workspace resolver and
annotationUseat scale; - excalidraw — a large
.tsxReact corpus: JSX invocations,anydensity,import typeerasure, path aliases.
No second TypeScript extractor exists, so the oracle rule does not apply; the resolution-rate categorisation, the
<lib>stub check and the self-hosting graph query are the substitutes. - TypeScript 4.9's own
npx codegraph-typescript --src . --out model.jsonl on any machine with
Node ≥ 22 — the same floor as codegraph itself. The published package
bundles the extractor into one file (tsup, platform: node, typescript
external) and declares typescript as its one dependency, so an install is
the compiler plus a few hundred kilobytes. bin/codegraph-typescript at the
repository root runs the same bundle from a checkout, and is what
test.sh --ts smoke-tests and what codegraph snapshots --extractor is
pointed at.
A Node single-executable application (SEA) per OS — the C# distribution
shape — is deferred, with the trap named now so it is not rediscovered: a SEA
embeds the bundle but not the typescript package's lib/*.d.ts files, and
ts.getDefaultLibFilePath then points into a directory that does not exist,
which binds no standard library and passes every test run from sources. Doing
it means embedding the lib files as assets and a custom CompilerHost that
serves them from memory — a build-and-host change, not a model change — and
it is worth doing only once someone needs the extractor on a machine without
Node, which today is nobody who can run the rest of the pipeline — and the
desktop distribution (§15) does not create that someone: codegraph-typescript
is a Homebrew formula that depends on Homebrew's node, so the machine has
Node without the user managing it, and this package stays exactly as it is.
Wiring into the repo's scripts and CI:
build.sh --ts(alreadypnpm -r build) listsextractors/typescript/distin its artifact report;test.sh --tsadds the built-bincmpagainst the snapshot..github/workflows/ci.yml: theverifyjob already builds and tests the package as part of the workspace; atypescript-smokematrix runs the built bin onfixtures/typescript/srconubuntu-latest,macos-latestandwindows-latestandcmps the output with the snapshot — Windows is the one that matters (\paths, CRLF checkouts, case-insensitive walks), andactions/setup-nodemakes it a two-step job. On av*tag the package is published to npm with provenance (npm publish --provenance) beside the C# binaries on the release.README.md+CLAUDE.mdarchitecture block gainextractors/typescript/;docs/typescript-extractor.mdmirrorsdocs/csharp-extractor.md(run withnpx, from a checkout, the flags, reading the summary, troubleshooting — the tsconfig-conflict and<lib>-empty symptoms first).
| Hazard | Where it shows | Guard |
|---|---|---|
typescript inlined into the bundle |
built bin binds no lib: every Array is <unresolved>; sources pass |
tsup external: ["typescript"]; the <lib> test runs against the built bin |
\ path separators and drive letters |
Windows anchors, definedIn, module keys |
ts.sys paths normalised with / before they enter a key; paths.test.ts on a \-joined input; the Windows smoke job |
| case-insensitive file systems | macOS/Windows: the checker's useCaseSensitiveFileNames differs, Foo.ts/foo.ts cannot coexist, enumeration order differs |
ordinal sort of walked paths; the host is created with useCaseSensitiveFileNames: true so keys keep the written case; DeterminismTest shuffles the walk |
\r\n sources |
Windows checkouts, .gitattributes-less corpora |
ts.getLineAndCharacterOfPosition is EOL-agnostic; the sloc scanner test with both endings; the CRLF determinism case |
tsconfig conflicts in a monorepo |
two roots with incompatible paths |
first root wins, the conflict named on stderr; --tsconfig overrides; a fixture with two roots |
Intl/locale-sensitive sorting |
canonical order on a machine with a different locale | comparisons by UTF-16 code unit (< on strings), never localeCompare; an eslint rule forbidding localeCompare in the extractor |
JSON.stringify of lone surrogates and non-BMP names |
byte identity with core | it IS core's encoder, natively — the one hazard this extractor lacks |
| memory on very large programs | the checker holds every type; excalidraw and nest are fine, vscode-sized corpora are not |
--max-old-space-size documented; Program created with skipLibCheck and no emit; an explicit "one program per --src root" flag is a later enrichment if a corpus needs it |
- M13a — profile v2 + skeleton + contract ✅ (2026-09-08):
feat(core): typescript profile v2(§14.2) with the METAMODEL §1.1 clause for nameless top-level entities; the workspace package with its boundary test; the walking skeleton (§14.6 first bullet) →fixtures/typescript/expected/model.jsonlbyte-identical to core's encoder and profile-valid with zero issues; the core gate;snapshots --extractorrunning.jsundernode;bin/codegraph-typescript;test.sh --tswith the built-bincmp. - M13b — the model ✅ (2026-09-08): members (methods, accessors as
#get/#set, statics as#staticbeside an instance twin, constructors, properties incl. parameter properties, enum members with computed values, parameters, destructured bindings, locals as#local:name:line:column, nested functions as#fn:), nameless invocables byline:column, bound object literals and class expressions, overloads folded to the implementation, cross-kind merging by rank; every edge kind incl.annotationUsewith written arguments named after the factory's parameters andthrowsat the site; JSX elements as invocations; accessor writes on the setter;sloc+cyclomatic; constant-shaped values (4 * 25unevaluated); the full 23-file fixture and its README; the fixture in the analyzer, city, navigator and CLI suites; every CLI command verified on it. Snapshot: 193 entities / 127 edges. - M13c — self-hosting + audit + distribution ✅ (2026-09-09):
-
Codegraph on codegraph (
--src packages --src extractors/typescript): 24 887 entities / 51 255 edges in 12 s,validateclean, 96.5 % resolution, ZERO<unresolved>, zero duplicates, zero unclosable edges; 168 imports resolved to workspace packages by name. The architecture rules of CLAUDE.md hold as graph queries (test/self-hosting.test.ts): no dependency cycle crosses a package;vizandnavigator-uireach their model packages only through imports and through TYPE-space entities (interface members ARE type-space — the audit's first correction to the profile);@openrouter/sdkis imported bypackages/llm/src/openrouter.tsalone,threebyvizalone,reactbynavigator-uialone. City (355 stub buildings unmeasured, as designed) and navigator reviewed as screenshots. -
Three-corpus audit, every model
validate-clean:corpus files entities / edges rate <unresolved>any-receivers time microsoft/TypeScript 4.9.5 src/compiler(namespaces +/// <reference>,outFilestyle)69 52 341 / 120 127 97.4 % 3 ( Profile,Session,Stats— node types with no@types/node)2 684 21 s nestjs/nest packages(decorators, a monorepo importing itself by name)902 29 992 / 32 537 88.6 % 78 (fastify/express/class-transformer types, not installed) 20 745 12 s excalidraw packages+excalidraw-app(.tsx, path aliases)628 43 706 / 62 557 94.4 % 41 (react/mermaid/vitest types, not installed) 24 954 23 s All three ran with NO
node_modules: the any-receiver counts are the ceiling the profile note describes — every call into an uninstalled package is a call throughany, dropped and counted — and nest's 508 workspace-resolved imports are its own@nestjs/*packages reached bypackage.jsonname. excalidraw's 303 stub modules are mostly font and asset imports (./Excalifont-Regular-….woff2), honest imports of non-code. Six defects found and fixed by the audit: members of an unbound object literal (array elements, arguments) were entities and collided (2 260 re-keyings on codegraph itself); a parameter owned an arrow written in its default value; methods of an unbound literal keyed their parameters under the enclosing function; members of an unbound class expression (return class extends Base {}, nest's dynamic modules) surfaced as members of the enclosing method and aborted the write with a dangling constructor; overloaded signatures with no implementation (an interface's, an ambient function's) each emitted parameters (74 on the compiler); a class expression bound to a PROPERTY (static readonly ConfigProxy = class …) was both a property and a class. Plus the safety net the contract asks for: an edge the model cannot close is dropped and counted (unclosable), never written dangling and never an abort. -
Distribution:
typescript-smokein CI runs the built bundle on Ubuntu, macOS and Windows andcmps the fixture snapshot;npm-publishon av*tag publishescodegraph-typescriptwith provenance when the repository holdsNPM_TOKEN(skipped, not failed, otherwise);docs/typescript-extractor.md; README and CLAUDE.md name the extractor.
-
Definition of done: fixtures/typescript/expected/model.jsonl byte-identical
to core's encoder and profile-valid with zero issues; the same bytes from the
built bin on Ubuntu, macOS and Windows runners; codegraph's own model
validate-clean with its package boundaries recovered as a graph query; the
audit numbers in the profile notes; ./test.sh and CI green with the
TypeScript fixture in every per-fixture suite.
Everything a user needs, from Homebrew, with nothing to manage on the machine — no JDK, no .NET SDK, no Node of one's own. The app is ONE install; every language extractor is its OWN install, taken only by those who need it:
brew install --cask defsquare/tap/codegraph # Codegraph.app + `codegraph`
brew install defsquare/tap/codegraph-java # one per language, as needed
brew install defsquare/tap/codegraph-csharp
brew install defsquare/tap/codegraph-typescript
brew install defsquare/tap/codegraph-elixir # and every extractor to come
The cask puts Codegraph.app in /Applications and codegraph on PATH;
each formula puts one codegraph-<language> command on PATH. The app opens
a folder and shows the navigator with the city as a tab — the codegraph serve page — after running the right extractor, which it FINDS on the
machine rather than carries; a folder whose language has no extractor
installed is answered with the one brew install line that fixes it. The
CLI is the backend: nothing is rewritten, the pipeline of §7, §11, §12 and
the navigator is called through a long-lived process instead of one command
line.
WHY SPLIT. Every extractor carries its own runtime — a GraalVM image for
Java, the .NET single-file host for C#, the BEAM for Elixir — and each is
tens of megabytes per architecture that a user of one language never needs.
One cask with every extractor inside would grow with each Phase 7+ language
and re-download all of them on every brew upgrade of any one; one formula
per extractor keeps the app download at the SEA and the shell, lets an
extractor release on its own cadence, and is the shape Homebrew already has
for a command on PATH. The extractors were designed as separate processes
under one command-line contract (§13.5, schemas/README.md §8) exactly so
this could be true: the app needs to know nothing about them but that
contract and the file extensions they claim.
Locked decisions, each with its reason (the delta table in §18 repeats them):
- Tauri is the shell, not a runtime. Window, native menu, folder dialog,
drag-and-drop, sidecar lifecycle, bundling, signing and notarization. No
business logic in Rust: the analyzer, the city and the navigator stay
TypeScript, and the Rust crate never reads a
model.jsonl. A Rust rewrite of anything underpackages/is out of scope by construction. - The Node side is ONE binary, ONE program, a Node single-executable
application (SEA) built from the CLI bundle with the two frontends embedded
as assets. It IS
codegraphin a terminal and the daemon behind the app. It carries no extractor: the TypeScript extractor is its own install like every other (§15.1), so the image has no dispatch by name and no compiler inside — §15.3. - One install per extractor, each a formula in our tap.
codegraph-java,codegraph-csharp,codegraph-typescript,codegraph-elixir… are formulae indefsquare/homebrew-tap, one command onPATHeach, per architecture where the runtime is native, generated by that extractor's own release job. The app bundles none of them and discovers what is installed (§15.4); a language without its extractor is a message naming the install line, never a silent failure. App and extractors version independently and meet on the interchange contract (schemaVersion, schemas/) — the same boundary that lets a Go or Python extractor exist without touching the app. - The page comes from the daemon, same origin. The webview navigates to
the daemon's loopback URL and fetches
navigator.json/city.jsonbeside it — exactly whatcodegraph servedoes today (?src=/?city=and the sibling routes, unchanged). No Tauri IPC in the page, sonavigator-uinever imports@tauri-apps/*(the boundary test scans for it) and CORS betweentauri://localhostand127.0.0.1never arises.frontendDistis a one-line "starting…" page the shell replaces once the daemon has printed its port. - A capability URL, not an open port. The daemon binds
127.0.0.1:0and prints one JSON line{ "port", "token" }on stdout; every route lives under/<token>/. A code model is sensitive (§7's--hostannouncement exists for that reason), and on loopback any page in any browser could otherwise read it. TheOriginheader, when present, must be the page's own. - The daemon dies with its parent. It exits when stdin reaches EOF, so a crashed or killed shell cannot leave an orphan holding a model in memory. The shell also kills the child on window close; both rules, not one.
- Extractors are a registry the shell hands over, as data. Each entry is
{ name, path, extensions[], launch, env?, install? }— the §13.5 contract is the only thing the daemon knows about any of them, and the language-blind rule of §13.5 ("there is nocodegraph extract") holds: the daemon runs a registry entry, it never names a language. Detection is a census of file extensions againstextensions[]; a tie is a question the page asks, never a guess. The shell writes the registry from what it FINDS (§15.4): an entry it knows but did not find has nopathand carriesinstall, the line to type, and a tree only such entries claim is answerednot-installedwith that line. - Homebrew: a cask for the app, a formula per extractor, one tap, updates
through
brew upgrade. Tauri produces a.appin a DMG, which is what a cask installs and a formula cannot; an extractor is a command onPATH, which is what a formula installs — from a prebuilt per-architecture binary, legitimate in our own tap (homebrew-core refuses vendored binaries; a tap does not). The Tauri updater plugin stays off: two update paths disagree eventually. Linux ships the Tauri.deb/AppImage on the GitHub release and Windows the NSIS installer — both built by the existing runner matrix, both outside brew's scope, neither a gate of M14; the extractors' binaries for those OSes are already on their own releases (§13.7, §13.10, §14.7).
M13 landed on main before this phase. With one install per extractor, the
TypeScript extractor stays exactly the npm package M13 shipped — and its
Homebrew form is a formula that installs that package under Homebrew's own
node, the way Homebrew ships every Node command (typescript, eslint…):
| M13 decision | M14 consequence |
|---|---|
the extractor is an npm package with typescript external, run with npx codegraph-typescript (§14.7) |
KEPT, and it is the Homebrew form too: Formula/codegraph-typescript.rb has depends_on "node", installs the published tarball into libexec with std_npm_args and links bin/codegraph-typescript. A ~12 MB download (bundle + compiler) on top of a node Homebrew already shares with everything else, instead of a second 110 MB Node image or 12 MB folded into the app for every user |
typescript stays OUTSIDE the bundle so the compiler finds lib.*.d.ts beside itself (§14.1, §14.7) |
unchanged — inside libexec/lib/node_modules/… the libs ARE beside the compiler; the §14.7 trap (a single file has no "beside itself") never arises because the extractor never becomes a single file. The <lib> built-bin test of §14.8 keeps its two runs |
the extractor never imports @codegraph/* (boundary test), and the Node side never learns a language (§13.5) |
unchanged, and now also a packaging fact: the SEA (§15.3) carries no extractor at all; the app reaches TypeScript as it reaches Java — a registry entry, a process |
snapshots --extractor runs a .js under node |
in the app the registry entry's path is the formula's bin/codegraph-typescript shim, launch: exec; node is Homebrew's, never the SEA's |
run(args, io, cwd) in main.ts, cli.ts a two-line program |
untouched |
| self-hosting (§14.6) | apps/desktop/src joins the --src roots of the self-hosting run, and the boundary query gains one row: @tauri-apps/* is imported under apps/desktop alone |
Sizes, so the split is measurable: the app cask downloads the SEA (~110 MB,
of which both frontends are 1.4 MB) plus the shell; codegraph-typescript
~12 MB plus Homebrew's node once; codegraph-java and codegraph-csharp
are each their native image per architecture, tens of megabytes a user of
another language never fetches.
Today serve takes models on argv and exits with the window. --app is the
long-lived form:
codegraph serve --app --data-dir <dir> --extractors <registry.json> [--host 127.0.0.1]
stdout: exactly one line, {"port":N,"token":"…"}, then nothing
exit: when stdin closes, or on SIGTERM
Routes, all under /<token>/:
| route | what |
|---|---|
GET / and the static page |
the navigator-ui bundle, from SEA assets (the vizAssetsDir/navigatorAssetsDir lookups of serve.ts gain an asset-backed branch; the checkout branch stays for pnpm users) |
POST /jobs { src, extractor? } |
detect (census) or take the named registry entry, run it under the §13.5 contract into <data-dir>/models/, then build navigator.json + laid-out city.json exactly as serveCommand does (cityBuildOf, navigatorOf); one job at a time, a second POST while one runs is 409 |
GET /jobs/current (SSE) |
the extractor's stderr progress lines + the build phases, then done or failed with the extractor's exit code and the last stderr lines |
GET /navigator.json, GET /city.json |
the current project's artifacts, the routes the page already fetches |
GET /recent |
the last projects, <data-dir>/recent.json; the page's empty state lists them |
model.db and the artifacts live under --data-dir (the shell passes the OS
app-data directory, ~/Library/Application Support/codegraph on macOS), so a
reopened folder skips extraction when the tree is unchanged — the M7 cache
doing its job. The page's empty state (the existing Loader) learns the
--app form: recents, a progress view fed by the SSE route, and the
instruction "File › Open… (⌘O), or drop a folder on the window" — the shell
owns the dialog. Everything above is tested under Vitest with a fake extractor
(a script that copies a fixture into --out and prints two progress lines),
including the capability token (a request outside /<token>/ is 404, never
401 — the route does not exist), the stdin-EOF exit, and the 409.
- CommonJS, one file. A SEA's main script is CommonJS; the CLI bundle is
ESM with a dynamically imported
explainchunk. A second tsup target —format: cjs,splitting: false,platform: node, the workspace packages inlined — feeds the SEA; the ESM build stays whatpnpmusers run.node:sqliteis a built-in and needs nothing. No extractor is in the image (§15.1), so there is no dispatch on the invoked name and no compiler to inline: the entry is the CLI's, unchanged. - Assets.
sea-config.jsonlists the two frontenddist/trees andpackage.json(for--version);serve.ts'snavigatorAssetsDir/vizAssetsDirgain an asset-backed branch behind the same resolver interface the checkout paths use —node:sea'sisSea()decides, and the tests exercise one seam with two implementations. - CI. A
seamatrix on the five existing runners (the Java native job's shape: no cross-compilation, macOS signs on macOS) builds the image and gates it three ways:codegraph --versionprints the package version,codegraph analyzeon the Java fixture byte-identical to the ESM build's output, andserve --appanswering/<token>/navigator.jsonon the fixture under the fake extractor of M14a — the page's assets served from the image, not from a checkout. - macOS.
postjectbreaks Node's signature:codesign --remove-signaturebefore injection, an ad-hoc signature after (arm64 refuses to run an unsigned Mach-O at all), and the real signature comes with the bundle (§15.5).
-
ONE sidecar in
tauri.conf.jsonbundle.externalBin: the SEA, one file per Rust target triple. The release job renames what theseamatrix produces:RID today triple osx-arm64 aarch64-apple-darwin osx-x64 x86_64-apple-darwin linux-x64 x86_64-unknown-linux-gnu linux-arm64 aarch64-unknown-linux-gnu win-x64 x86_64-pc-windows-msvc Two DMGs, arm64 and Intel, rather than a universal binary: the SEA is per triple and a universal
.appwould carry both. No extractor rides in the bundle. -
Extractor discovery. The shell ships a CATALOGUE,
extractors.jsonin its resources — one row per extractor codegraph knows:{ name, extensions[], command, install }(codegraph-java,[".java"],brew install defsquare/tap/codegraph-java; and so on for csharp, typescript, elixir…). On launch it looks eachcommandup onPATHand in the Homebrew prefixes (/opt/homebrew/bin,/usr/local/bin, the Linuxbrew prefix), then writes the daemon's registry: found →{ name, path, extensions, launch: "exec" }; not found →{ name, extensions, install }with nopath. A settings entry lets a user point a row at a binary anywhere (a checkout'sbin/codegraph-typescript, a jar withlaunch: java). The catalogue is data: a new extractor is one row, no Rust. -
The daemon's side of that (a small M14c change to
registry.tsandjobs.ts): an entry withoutpathis legal, counts in the census, and when the only claimants of a tree are such entriesdetectanswersnot-installed—POST /jobs→422 { error: "not-installed", extractors: [{ name, files, install }] }— and the page shows the line to type with a "Check again" that asks the shell to re-run discovery. A tree claimed by one installed and one missing extractor is stillambiguous, and the missing candidate is offered with its install line rather than hidden. -
On launch: discovery, write the registry, spawn
codegraph serve --app --data-dir <app data> --extractors <registry>as a sidecar with stdin held open, read the one stdout line, navigate the webview tohttp://127.0.0.1:<port>/<token>/. On window close: close stdin, kill the child. The Rust crate is that, the discovery, a menu (File › Open…, Open Recent) and the drag-and-drop handler, each of which does onePOST /jobs— a few hundred lines with no dependency on the model. -
The WebGL gate. WKWebView is not Chrome. Before anything else in M14c,
tauri devon the fineract city (the NV milestone's 100 MB artifact) at user-facing angles, screenshots reviewed, frame time measured; a failure here changes the plan (a Chromium-based shell) and is cheaper to learn first. WebKitGTK on Linux is known to be fragile with WebGL, one more reason Linux is not a gate.
tauri buildsigns and notarizes from the Apple Developer credentials in CI (APPLE_CERTIFICATE,APPLE_CERTIFICATE_PASSWORD,APPLE_SIGNING_IDENTITY,APPLE_ID,APPLE_PASSWORD,APPLE_TEAM_ID) with hardened runtime, and signs the one sidecar with the same identity. The entitlements file is the bundle's one file and carries the JIT trio —com.apple.security.cs.allow-jit,com.apple.security.cs.allow-unsigned-executable-memory,com.apple.security.cs.disable-library-validation— because the SEA (V8) needs it. If the bundler turns out not to apply the entitlements to the sidecar, CI signs it itself withcodesign --options runtime --entitlementsbeforetauri build— a step to verify on the first notarized build, not to assume.- Two things to confirm on the first signed build, each a CI assertion
afterwards:
spctl --assessaccepts the DMG's.appon a clean machine, andcodegraph --versionruns from/Applications/Codegraph.app/Contents/ MacOS/codegraphunder the hardened runtime. - The extractor binaries are formula downloads, not bundle contents.
Homebrew fetches them with
curl, which sets no quarantine attribute, so Gatekeeper does not assess them; arm64 still refuses an unsigned Mach-O, which the .NET publish and GraalVM both satisfy with an ad-hoc signature today. The macOS release jobs ofcodegraph-javaandcodegraph-csharpnevertheless sign with the Developer ID when the identity is present (the .NET single-file host wants the JIT trio too, per Microsoft's notarization guidance) — a clean-Macbrew installof each is the assertion that decides whether notarization of a bare binary is also needed, and the answer goes in the profile notes. The osx C# publish must be truly ONE file (only the linux-x64 dist exists on the dev machine; .NET refuses to embed native libraries in a single file on macOS, and a stray.dylibbeside the host is not somethingbin.installcarries). defsquare/homebrew-tap,Casks/codegraph.rb:arch arm:/intel:, the DMG URL per architecture from the app's GitHub release,sha256per architecture,app "Codegraph.app", ONEbinarystanza —Contents/MacOS/codegraph.auto_updates false, alivecheckon the app's release tags.Formula/codegraph-java.rb,Formula/codegraph-csharp.rb:on_arm/on_intel(andon_linux) blocks with the binary's URL andsha256from THAT extractor's release,bin.install, atest dothat runs--versionand extracts a two-file fixture written by the test itself;livecheckon that extractor's tags.Formula/codegraph-typescript.rb:depends_on "node",urlthe npm registry tarball,system "npm", "install", *std_npm_args,bin.install_symlink Dir["#{libexec}/bin/*"]— Homebrew's standard Node-package formula, the<lib>files beside the compiler as §14.7 requires. Every future extractor (codegraph-elixir…) adds one formula in the same shape, in its own release job.- Generated, never edited by hand. Each release job — the app's, and each
extractor's — gains a step that rewrites ITS cask or formula with the new
version and hashes and pushes to the tap with a deploy key. Versions are
independent: the cask and the formulae bump on their own tags, and
compatibility is the interchange contract — the daemon validates every
model it opens exactly as
servedoes, and aschemaVersionthe app does not know is afailedjob whose message namesbrew upgrade codegraph(or the extractor), not a crash.
- M14a — the daemon ✅ (2026-09-16).
serve --app [--data-dir DIR] [--extractors FILE]inpackages/cli/src/app/:registry.ts(the registry as data,launchinferred from the path assnapshots --extractordoes, the extension census,detect→ one | ambiguous | none, the size-and-mtime tree fingerprint over the files the chosen extractor would read),store.ts(<data-dir>/models/<name>-<hash>/withmodel.jsonl, themodel.dbopenCachebuilds beside it, the two artifacts andproject.json;recent.json; the platform default data dir),jobs.ts(one job at a time;started/phase/progress/done/failedevents replayed to a late subscriber; the extractor skipped when the fingerprint has not moved, the build skipped when the build key has not either — a reopen from the recents took 59 ms) anddaemon.ts(the 32-hex token minted per launch,/<token>/routes: the page,app,recent,navigator.json,city.json,POST jobs→ 202/409/422/404,jobs/currentSSE that saysidleand stays open; anOriginthat is not the page's own is 403, anything outside the token is 404; stdin EOF and SIGTERM close the server and kill a running extractor). Two things the plan did not name and the daemon does: aPOST jobswhosesrcis amodel.jsonlbuilds the page with no extractor, and an empty registry is legal (the dev loop from a checkout needs neither a registry nor a JDK). navigator-ui:app-mode.ts(probe by thecodegraph.app/1kind, typedsubmitJoboutcomes, the event reducer) andAppHome— the recents, "File › Open… (⌘O), or drop a folder on the window", a path field as the browser dev loop, the three phases with a window on the extractor's stderr, the "which extractor?" question, failures with the exit code and the last stderr lines — plus an "Open…" header button and the@tauri-apps/*boundary test. Tests: 19 registry, 17 daemon over real sockets againsttest/fake-extractor.mjs(the §13.5 contract: copies a fixture, two stderr lines,FAKE_EXTRACTOR_FAIL/SLOW_MS/LOG), 3 process tests (one stdout line through a real pipe, exit 0 on stdin EOF, exit 2 on a bad registry before binding), 8 serve-command tests, 17 app-mode tests. Driven headlessly on the Java fixture (fake extractor) and a mixed tree (the real TypeScript extractor after the question), screenshots reviewed. - M14b — the SEA ✅ (2026-09-16).
packages/cli/tsup.config.tsgains a second target:src/sea-entry.ts(the samerun, no top-level await) →dist-sea/codegraph.cjs, CommonJS,splitting: false,noExternal: [/.*/],shims: true, 2.9 MB with every workspace package and zod inlined and only Node's built-ins external.src/sea.tsowns the three image facts throughprocess.getBuiltinModule("node:sea")— a staticimport "node:sea"came out of the bundler asrequire("sea"), which the prefix-only builtin does not answer —isSeaImage(),seaAsset(key), andnodeCommand()(nodeon PATH inside the image,process.execPathin a checkout; used bysnapshots --extractorand the registry'slaunch: node).src/assets.tsis the one seam:FrontendAssets { label, read(relative) }withdirectoryAssets(jailed) andkeyedAssets/seaFrontendAssets(prefix)(the key space is the jail);serve.ts'svizAssets()/navigatorAssets()return one or the other,serveStaticand both servers read through it, andversion.tsreads thepackage.jsonasset before the file beside itself.scripts/sea-config.mjsGENERATESsea-config.json(every file of bothdist/trees asviz/…andnavigator-ui/…, pluspackage.json;useCodeCache),scripts/sea-build.mjsdoes the seven steps (blob, copy this Node, macOScodesign --remove-signature,postjectwith the sentinel fuse and theNODE_SEAMach-O segment, ad-hoc sign) intodist-sea/<rid>/codegraph[.exe],scripts/sea-smoke.mjsis the three-gate acceptance (--version= the package's,analyzedeps/cycles/coupling byte-identical to the ESM build,serve --appunder the fake extractor serving index.html AND its hashed script from the image, openingfixtures/java/src,navigator.json+ laid-outcity.json, exit 0 on stdin EOF)../build.sh --ts --sea,test.sh's phase when the image exists, the five-runnerseaCI job (codegraph-cli-<rid>artifacts,codegraph-<rid>on the release). Verified on linux-x64: a 131 MB image, all 13 gate checks green,node:sqliteand the model.db cache working inside it. Tests: the asset seam with two sources through one server (27), the config generator,versionOf; every existing suite rewired toFrontendAssets. - M14c — the shell ◐ (2026-09-16, written on a Linux machine without
WebKitGTK or a display: everything below the Tauri crate is tested here,
the crate itself is compiled by CI's
desktopjob, and THE WEBGL GATE HAS NOT BEEN RUN — it is the first thing to do on a Mac, per §15.4). The daemon's side:registry.tsaccepts an entry withoutpathwhen it carriesinstall; the census counts its extensions;detectanswersnot-installed{ extractors: [{ name, files, install }] }when only such entries claim a tree (or the chosen one is missing), and an installed + missing pair isambiguouswith the missing candidate carrying its install line;POST /jobs→ 422not-installed;/applistsinstalledandinstallper extractor; the registry is a SOURCE (registrySource, mtime-cached) asked on every request, so the shell's rewrite after a rescan is seen without a restart and a rewrite that does not parse keeps the last good one. The page: the extractors line marks "not installed", theNotInstalledpanel shows each install line (selectable) with "Check again" (a plain retry — no IPC), the "which extractor?" question shows a missing candidate's install line instead of a button.apps/desktop/: a Cargo workspace —discovery/(codegraph-discovery: the catalogue{name, extensions, command, install},search_dirs= PATH then$HOMEBREW_PREFIX/bin,/opt/homebrew/bin,/usr/local/bin, Linuxbrew;find_command(executable files only,.exeon Windows),discoverwithsettings.jsonoverrides (launch inferred for a jar/script), the registry JSON round-tripping the daemon's contract, atomicwrite_registry; 8 tests) andsrc-tauri/(daemon.rs: spawn the sidecar beside the executable with stdin held open, read the one line,POST jobs/GET recentoverureq, refusals as one sentence with the install line, shutdown = close stdin + kill;lib.rs: discovery → registry → spawn →navigatethe webview; File › Open Folder… (⌘O, native dialog), Open Recent (rebuilt fromrecent), Rescan Extractors; drag-and-drop; rescan + menu refresh on every window focus; close → shutdown; a refusal from the shell's own open is a native dialog).tauri.conf.json: one window, one sidecarbinaries/codegraph, the catalogue as a resource, the JIT-trio entitlements;capabilities/default.jsonlists the shell's permissions and deliberately noremoteorigin — the page has no Tauri API at all.scripts/sidecar.mjscopiesdist-sea/<rid>/codegraphtobinaries/codegraph-<triple>; icons fromtauri icon;dist/index.htmlthe one-line starting page;@codegraph/desktopin the pnpm workspace with no-opbuild/testand explicittauri:dev/tauri:build. CIdesktopjob (ubuntu with the WebKitGTK packages, macos): sidecar from theseaartifact,cargo fmt --check,cargo test --workspace. Verified here: the new daemon/registry/page tests (824 CLI tests green), 8 discovery tests,cargo fmt, lint; a headless run of the not-installed path — the home says "elixir — not installed", an Elixir tree shows the install line with Check again, the registry rewritten (as the shell does on focus) and Check again opens it without a restart, screenshots reviewed. NOT verified here: the Tauri crate compiling (no WebKitGTK),tauri dev, the screenshots on the Java fixture and fineract, the WebGL frame time. Two rough edges to revisit on a Mac: an ambiguous folder opened from the menu/drop is a dialog that sends the user to the page's path field (the page cannot be told what the shell posted without IPC), and thecodegraph://rescannedevent is emitted but nothing listens. - M14d — signed, notarized, in the tap ◐ (2026-09-16; the pipeline and
the generators, not yet a release). ci.yml
desktop-bundleonmacos-latest(osx-arm64) andmacos-15-intel(osx-x64): the sidecar from theseaartifact, the Developer ID certificate imported into a throwaway keychain, the sidecar signed by CI itself with hardened runtime and the JIT-trio entitlements BEFOREtauri build(the plan said not to assume the bundler does it),tauri build --bundles dmg --target <triple>(Tauri signs the app and notarizes fromAPPLE_ID/APPLE_PASSWORD/APPLE_TEAM_ID), then the §15.5 assertions as CI facts —codesign --verify --deep --strict,allow-jitpresent on the embeddedcodegraph,spctl --assess— and the DMG renamedCodegraph-<version>-<rid>.dmg. Every signing step is conditional onAPPLE_SIGNING_IDENTITY: without the secrets the job bundles ad hoc so a fork or a PR proves the bundling, and the signed-only assertions are skipped, never faked. Thereleasejob now needs it andnpm-publish, attaches the two DMGs, and gains the tap step:scripts/homebrew/render.mjsrendersCasks/codegraph.rb(arch arm/intel, one DMG URL + sha256 each,app+ ONEbinarystanza,auto_updates false,livecheckgithub_latest, a zap of the app data dir),Formula/codegraph-java.rbandcodegraph-csharp.rb(on_macos/on_linux×on_arm/on_intelwith the asset URL and sum from the release's SHA256SUMS,bin.installrenaming the asset, atest dothat extracts a one-file tree) andFormula/codegraph-typescript.rb(the npm tarball ondepends_on "node",std_npm_args, rendered only when the registry answers for the version), then clones the tap over an SSH deploy key and pushes "codegraph " — skipped, not failed, withoutHOMEBREW_TAP_DEPLOY_KEY. A missing sum aborts the render before any file is written. 7node --testcases pin the templates to the release's asset names (atest.shphase).docs/distribution.mdnames every secret, what each job puts on the release, and how to render by hand. Departure from the text above: all four tap files are rendered from ONEv*tag by the one release job, not by per-extractor release jobs — the repository has one release workflow today; independent extractor cadence means per-extractor tags, deferred until an extractor needs a release of its own. 2026-09-17: the tap is https://github.com/defsquare/homebrew-tap — public, already carrying Defsquare'sdatagraphandspecyformulae under the same "generated, pushed by the tool's own release" rule, and needingbrew trust defsquare/taponce on newer Homebrew; the generator renderscodegraph-elixirtoo (Apple silicon + Linux x64, the platformselixir-nativebuilds) and the release job attaches the Burrito binaries ascodegraph-elixir-<rid>. NOT done, and not doable from here: the tap deploy key and the Apple credentials are not in the repository's secrets (creating credentials is a maintainer's act —docs/distribution.mdhas the three commands), and no tagged release has run — so no notarized DMG exists, the tap holds no codegraph file yet, and the clean-Mac install checks of the definition of done are outstanding.
Definition of done: a clean Mac with Homebrew and nothing else runs the cask
line, opens the app, is told on a Java folder which line installs
codegraph-java, runs it, and then opens a Java, a C# and a TypeScript
folder from the app and sees the city and the navigator; codegraph analyze,
codegraph-java, codegraph-csharp and codegraph-typescript work in a
terminal from those installs, each reproducing its fixture snapshot byte for
byte; the SEA job is green on all five runners; pnpm -r test green with the
daemon and resolver seams covered; the WebGL screenshots reviewed; the app
download carries no extractor.
Motivation: the fourth real extractor is the first for a dynamic, macro-first
language on a VM with no class library of types — the case the trait design
was built for (§2, the Clojure fn-var) and the one no shipped extractor has
exercised. Elixir brings three things the pipeline has not modeled: a
defmodule that is at once the compilation unit, the namespace and the type
(a struct, a behaviour, a protocol are all modules); dispatch that is
overwhelmingly dynamic by design (protocols, behaviours called by OTP, message
passing) and must be reported honestly rather than guessed; and code that does
not exist before macro expansion (use, @derive, a Phoenix router, an Ecto
schema) — the Clojure profile's generated provenance, for real. Its legacy is
of the non-compilable kind too: a Phoenix 1.3 app whose deps/ no longer
resolves and whose mix compile fails at the first use Ecto.Schema.
What is the Spoon / Roslyn / tsc of Elixir? The compiler again — but split in two, and the split is the design:
Roslyn / ts |
Elixir | Role |
|---|---|---|
SyntaxTree |
Code.string_to_quoted/2 → the quoted AST ({form, meta, args}), columns: true, token_metadata: true (do/end/closing positions), literal_encoder for literal positions |
syntax with positions; the same data structure every macro receives |
SemanticModel / TypeChecker |
none before expansion. Binding happens in :elixir_expand, reachable only by compiling, through compilation tracers (Code.put_compiler_option(:tracers, …): remote_function, local_function, imported_function, alias_reference, require, import, struct_expansion, on_module) |
exact calls after macro expansion — Spoon-grade, and unreachable without every dependency present |
the BCL ref pack / lib.*.d.ts |
the Elixir standard library and Erlang/OTP: a module and export table generated at build time from the building Elixir/OTP (Application.spec(:elixir, :modules), :code.all_available/0, Module.exports/1) and embedded in the extractor |
what Enum.map/2, Kernel.is_nil/1, :ets.lookup/2 resolve to on a machine with nothing installed |
IErrorTypeSymbol |
a local call that binds to nothing (a macro-injected import the AST cannot see) | binding failure, tolerated, counted, never fatal |
So the baseline is a lexical resolver over the parser's AST — the alias /
import / require / use scope stack, nested-defmodule auto-aliases,
__MODULE__, the embedded OTP table — and it needs nothing but source. It is
"noClasspath" by construction: a file that parses is a file that extracts. The
tracer is the optional enrichment (--trace, §16.5), the --references slot
of §13 and the --deps idea of §14: it needs a project that compiles, and when
it has one it recovers what the baseline honestly could not.
The alternatives, and why they are not the baseline:
| Candidate | What it is | Verdict |
|---|---|---|
mix xref |
the compiler's own dependency graph (mix xref graph, compile/export/runtime edges between modules) |
not an extractor — it needs a compiled project, knows modules only (no functions, no anchors, no provenance), and says nothing a tracer does not. It becomes an oracle (§16.6): on a corpus that compiles, the baseline's module layer must reproduce it |
| tree-sitter-elixir driven from TypeScript | a parser with a CST | rejected — it re-implements the Elixir parser's desugaring (do-blocks, keyword lists, multi-clause heads, sigils, operators as calls) to reach the quoted form the language defines, and can never reach tracers. The compiler's own parser is the reference and is a library call away |
| ElixirSense / ElixirLS / Expert's Spitfire | editor-oriented analysis (error-tolerant parsing, completion metadata) | not a baseline — designed for one file with a warm project, not one pass over a corpus; Spitfire's error tolerance is a later enrichment for files the compiler's parser rejects (§16.8) |
Code.compile_file / Code.eval_* on corpus files |
running the corpus | never — evaluating corpus code is the thing the pipeline refuses (§5.2, §13); a mix.exs is read as source, not run |
Three principles, locked up front:
- The parser without the compiler — the noClasspath of Elixir. The
extractor never runs
mix, never reads_build, never loads a corpus module. It walks--srcfor*.exand*.exs(skipping_build,deps,.elixir_ls,node_modules), parses each file to its quoted form, and resolves names through a lexical scope stack plus the embedded OTP table. A module the corpus does not declare is a stub, not a failure; a file the parser rejects is skipped and counted, never fatal. Everything visible before expansion isdeclared; what only expansion produces isgeneratedwhen the language itself defines the expansion (use XcallsX.__using__/1;@derive Pdefines an impl) and absent otherwise, with the absence counted. - No metamodel intelligence, and the extractor is a BEAM program. It is
written in Elixir, on the language's own parser, and shipped as an escript
(the jar: needs Erlang/OTP on the machine, nothing else — an escript embeds
Elixir) and as a Burrito binary per OS (the GraalVM image: needs nothing).
It knows the Elixir id scheme, the kind→traits table and how to write bytes;
trait vocabulary, profile validation and closure live in
core, and the cross-language gate (packages/core/test/fixtures-elixir.test.ts) is where the two halves meet.codegraph snapshots --extractorand the M14 registry run it as a process under the §13.5 contract and learn no Elixir. - Byte-identity across OS and across install state. Two runs over one
unchanged corpus write the same bytes on Linux, macOS and Windows, from the
escript and from the binary; and because external modules are keyed by
module name under reserved modules (
<otp>,<deps>,<unresolved>, §16.3), never by a path underdeps/, the keys of a model do not depend on what is checked out beside it. Edge counts do (--depsresolves imported names through dependency sources, §16.5), and the fixture snapshot is produced with nodeps/at all.
extractors/elixir/ Mix project `codegraph_elixir` (escript name `codegraph-elixir`)
mix.exs deps: NONE at runtime (Elixir ≥ 1.18 for the built-in `JSON`); ex_unit + stream_data (property tests) for test
lib/codegraph_elixir/
cli.ex the extractor contract (schemas/README.md §8) + --deps, --trace
corpus.ex pass 0: walk (ordinal order), parse each file (`Code.string_to_quoted_with_comments/2`), the skipped-file count
otp.ex the embedded OTP table (modules + exports of the building Elixir/OTP), generated by `mix codegraph.otp_table` at build time
scope.ex the lexical resolver: alias / import / require / use stack, nested defmodule, __MODULE__, Kernel except:
whitelist.ex pass 1: every defmodule under the roots — membership, never a name prefix
ids.ex THE Elixir id scheme (§16.3): keys, escaping, arity
entities.ex pass 2
edges.ex pass 3
stubs.ex pass 4: <otp> / <deps> / <unresolved> stubs
measures.ex sloc (`:elixir_tokenizer`, the compiler's own lexer) + cyclomatic
literals.ex module-attribute constants, struct-field defaults, parameter defaults (`\\`)
model/ NaturalKey, Entity, Edge, canonical order (UTF-16 code units), JsonlWriter (`JSON.encode_to_iodata!`)
test/ ExUnit; the C# suite's names where the property is the same (snapshot, determinism, stub-discipline,
ids, schema-per-line, cli) + scope, arity, clauses, defimpl, dynamic-candidate
priv/otp/<elixir>-<otp>.etf the generated OTP table (Erlang term format), one per building toolchain; the fixture pins one
dist/<rid>/codegraph-elixir Burrito output, one per OS/arch (build.sh --elixir --native)
bin/codegraph-elixir the escript (build.sh --elixir)
fixtures/elixir/src/ the reference corpus (§16.6): `acme_order`, the Java, C# and TypeScript corpora's twin, Mix-shaped
fixtures/elixir/expected/model.jsonl
scripts/lib.shgainshave_elixir_extractor,ensure_erlang(Erlang/OTP ≥ 27 and Elixir ≥ 1.18 on PATH, or the instructions to install them withmise/asdf);build.sh --elixirrunsmix escript.build,--nativeadds Burrito;test.sh --elixirrunsmix testand then the built escript AND the binary (when present) onfixtures/elixir/srcwithcmpagainst the snapshot — the Java/C# shape. CI addselixir-test(erlef/setup-beam) andelixir-smokeon the three OS runners.- Nothing installed on the machine that runs the binary. An escript needs
escript(Erlang) on PATH — the jar's JDK. Burrito wraps a Mix release with its ERTS into one executable per target (Linux x64/arm64, macOS x64/arm64, Windows x64), cross-built from one host through Zig — thedotnet publishmatrix of §13.7. The--versionoutput names the Elixir and OTP versions the OTP table was generated from, and the header carries them as extra keys underextractor(the header schema allows it), so a model says which standard library it resolved against. codegraph snapshots --extractorlearns nothing new: an escript is a file with#!/usr/bin/env escriptand runs directly on Linux/macOS; on Windows the registry entry (§15) namesescript.exe <path>or the Burrito binary. The M14 cask gains a fifthbinarystanza and the registry a fourth entry (extensions: [".ex", ".exs"]).- The extractor's own
mix.exshas no runtime dependency:JSON(Elixir 1.18),Code,Macroand:elixir_tokenizerare the whole front end.stream_datais a test dependency, the fast-check of this side.
There is no Elixir profile yet; the first M15 commit is feat(core): elixir profile — the tenth, data before implementation (invariant 8; the profile
registry's "nine profiles" test becomes ten). Its shape follows two prior
decisions rather than inventing a third:
| Decision | Why |
|---|---|
the module (TModule) is the file, the TypeScript rule verbatim |
Elixir has no namespace: MyApp.Accounts.User is one atom, and nothing contains it but the file it is written in (invariant 5). The import layer stays file-level and comparable with TypeScript's (invariant 9); umbrella apps (apps/<name>/lib/…) fall out as directory districts with no extra concept |
a defmodule is a module kind carrying TType — one building per defmodule |
the module IS the type: a struct is %Mod{}, a behaviour is a module, a protocol is a module, @spec f :: Mod.t(). Making only defstruct modules types would leave most of a Phoenix app's city empty; a separate struct entity would draw two buildings for one thing |
no inheritance, no embedding |
Elixir has neither; the absence is profile information, as in Go and Clojure |
TWithLocalVariables deferred (optional on the profile, unemitted in M15) |
every match binds; emitting each = as a local multiplies entities by ten for no edge the analyzer reads today. The trait stays licensed so M15c can add it without a profile change |
Kinds, with the traits that identify them (TSourceAnchor, TComment,
TMetrics optional wherever it makes sense):
| kind | required traits | Elixir construct |
|---|---|---|
file |
TNamed, TModule, TWithChildren |
a source file; optional TWithInvocations, TWithAccesses — config/*.exs and mix.exs call at top level (the TypeScript module rule) |
module |
TNamed, TType, TWithChildren, TChildOf, TSourceAnchor |
defmodule; optional TWithImplements (@behaviour, defimpl, @derive), TAttachedTo (a defimpl block, attached to the type it implements for), TComment (@moduledoc) |
protocol |
as module |
defprotocol; its defs are callback children |
function |
TNamed, TInvocable, TWithParameters, TWithInvocations, TWithAccesses, TChildOf, TSourceAnchor |
def / defp, every clause folded into ONE entity (§16.3); defdelegate is a function whose body is one invocation; private: true rides as a pass-through key (the metamodel has no visibility, the contract tolerates the key) |
macro |
as function |
defmacro / defmacrop; the guard @doc reads; a call to it is an invocation whose expansion is invisible |
callback |
TNamed, TInvocable, TWithParameters, TChildOf, TSourceAnchor |
@callback, @macrocallback, a protocol's def — the interface-method analog, no body |
field |
TNamed, TStructural, TChildOf, TSourceAnchor |
a defstruct / defexception key; optional TWithValue (the default) |
attribute |
TNamed, TStructural, TChildOf, TSourceAnchor |
@name value for a non-reserved attribute — the module constant; optional TWithValue, TWithAccesses (its value may reference modules) |
parameter |
TStructural, TChildOf |
one per position of the folded function; optional TNamed (from the first clause that names the position, _-prefixed and bare _ excluded), TWithValue (a \\ default) |
Construct → what the model says, on top of that table:
| Elixir construct | model |
|---|---|
defmodule A.B do … end, nested defmodule C inside it |
two module entities, A.B and A.B.C, both children of the file (nesting is an alias, not containment); C is aliased inside A.B (the auto-alias rule) |
defimpl P, for: T (and for: [T1, T2]) |
a module named as the compiler names it (P.T), child of the file, attachedTo → T; an interfaceImplementation edge T → P, declared, anchored at the block — the Clojure implBlock shape (METAMODEL.md §7) with the name Elixir gives it. for: Any → the stub type <otp>/Any |
@behaviour B |
interfaceImplementation module → B, declared |
use X / use X, opts |
an import edge file → X's file (declared, form use) AND an invocation of X.__using__#1 (declared: use is defined as require X + X.__using__(opts)). What __using__ injects — @behaviour, imports, defs — is NOT modeled in the baseline and every local it would have bound is counted (§16.4); --trace recovers it as generated |
@derive P / @derive {P, opts} |
interfaceImplementation module → P, provenance generated, anchored at the attribute — the language defines the expansion (Protocol.derive/3), so the fact is safe to state and honest to mark |
alias A.B, alias A.{B, C}, alias A, as: X, import M (only:/except:), require M |
one import edge kind, file → M's file (or the stub module), with the written form as a pass-through key (alias/import/require/use) — the TypeScript "three ways to import, one edge kind" rule. Scoped forms inside a function apply to that function's body only |
M.f(a, b), M.f a, b, x |> M.f(b) (arity + 1), :erl_mod.f(x), __MODULE__.f(x), alias-resolved F.f(x) |
invocation → M.f#arity; M resolved through the scope stack; a corpus M → the declared function (defaults fold, §16.3); M in the OTP table → its stub; else a stub under <deps> |
f(a, b) (a local call) |
resolved in this order: a def/defp/defmacro of the enclosing module with that name and arity (defaults folded) → an explicit import M that exports it (a corpus M, or the OTP table) → Kernel / Kernel.SpecialForms (unless import Kernel, except: removed it) → the sole non-corpus import M that could provide it (attributed to <deps>/M.f#arity, counted as import-attributed) → two or more candidates: dropped and counted → nothing: dropped and counted as local-unbound — the macro-injected import, the baseline's stated ceiling |
&M.f/2, &f/2, &M.f(&1, x) |
reference → the function; the call happens elsewhere (the Clojure higher-order rule) |
%M{…}, %M{s | …}, %__MODULE__{} in expressions and patterns |
reference → M (struct expansion); each named key → access to M.<key> (isRead in a pattern, isWrite in an update or construction) — the ONLY static field access Elixir has; s.name is a runtime map access, dropped and counted |
@attr read inside a body |
access → the module's attribute, isRead |
children = [MyWorker, {Registry, keys: :unique}], Supervisor.start_link(children, …), any module atom in argument position |
reference → the module — how the supervision tree surfaces; a module in argument position is a value, not a call |
GenServer.call(__MODULE__, msg), .cast, Agent.get(__MODULE__, …), send(self(), …) |
invocation → the target module's handle_call#3 / handle_cast#2 / handle_info#2, provenance dynamic-candidate with every clause-folded handler as the candidate set and to the first — the one message-passing form that is statically honest, because __MODULE__ names the module. Any other first argument (a pid, a name, a variable): dropped and counted as dynamic-dispatch |
P.f(x) where P is a corpus defprotocol |
invocation → P.f#arity (the callback), provenance dynamic-candidate, candidates = every corpus defimpl of P's f#arity — the Go interface-call and Clojure multimethod rule; a protocol from <otp> (Enumerable, String.Chars) → an ordinary stub invocation, its impls unknown |
raise M, raise M, msg, reraise M, … |
throws → M (<otp>/RuntimeError for raise "text", <otp>/ArgumentError for raise ArgumentError); throw/exit are not exceptions and yield nothing |
@spec f(A.t()) :: B.t(), @type t :: %__MODULE__{…}, @typep, @opaque |
reference edges from the function (or module) to every named remote type's module; @type is not an entity in M15 (noted) |
@moduledoc "…", @doc "…", the # comment block directly above a definition |
comments (TComment) — the docs are the semantic comment, the # block is what the TypeScript rule takes |
defexception [:message, :code] |
a module with field children, TWithImplements (the Exception behaviour it declares) |
defdelegate f(a), to: M, as: :g |
a function f#1 with one invocation → M.g#1, declared — the delegation is written |
apply(M, :f, args), mod.f(x) with mod a variable, Module.concat(…), Code.eval_*, :erlang.apply |
dropped and counted as dynamic-dispatch — the profile's blind spot, stated |
a file Code.string_to_quoted rejects |
skipped, counted as unparsed, the path on stderr; the model carries the file record without entities |
Explicitly NOT extracted in M15, stated in the profile notes: definitions a
used macro injects (a Phoenix router's routes, an Ecto schema's fields, a
plug pipeline) and the locals that call them; callback implementations as
edges (handle_call/3 under @behaviour GenServer — the analyzer recovers
"implements callback" from interfaceImplementation + name/arity, as it does
for Java overrides); message passing beyond the __MODULE__ form; process
topology at runtime (Registry, :via tuples, dynamic supervisors); .erl
files in a mixed corpus (an Erlang profile is its own phase); @type entities.
module (file) ex:lib%2Facme_order%2Forder.ex path relative to root, escaped
external (OTP) ex:<otp>/Enum ex:<otp>/ets ex:<otp>/Any the embedded table: Elixir + Erlang/OTP, by module name
external (deps) ex:<deps>/Ecto%2EChangeset a module nothing under the roots declares
(a stub has no members: `Enum.map/2` is an invocation of `<otp>/Enum` — the C# fold)
module (defmodule) ex:lib%2Facme_order%2Forder.ex/AcmeOrder%2EOrder the atom as written, its dots escaped
function / macro …/AcmeOrder%2EOrder.total#1 …/AcmeOrder%2EOrder.create#2 name, then ARITY as the disambiguator — always present
defimpl module ex:lib%2Facme_order%2Fmoney.ex/String%2EChars%2EMoney the compiler's name for it
callback …/AcmeOrder%2EPricing.price#2 the same shape as a function
field / attribute …/AcmeOrder%2EOrder.lines …/AcmeOrder%2EOrder.@max_lines no arity: a 0-arity `lines#0` cannot collide
parameter …/AcmeOrder%2EOrder.create#2#param:attrs …#2#param:2 (an unnamed position, by ordinal)
- Escaping, the TypeScript rule with one more input.
/,#and%are percent-encoded in every path segment and name;.is the nesting separator inside a symbol, so a module atom's own dots are encoded (AcmeOrder%2EOrder) exactly as a TypeScript name containing.is. The rejected shortcut — leaving the dots and relying on the rule that alias segments start uppercase while function names do not — holds fordefmodule A.Band fails fordefmodule :"a.b", which is legal; the encoding is injective by construction, pinned by astream_dataround-trip property, and the module'snameis the atom as written, which is what the city and navigator display. - Arity is identity, always.
f/1andf/2are unrelated functions in Elixir, so the disambiguator is never absent on afunction,macroorcallback.def f(a, b \\ 1)declaresf/1andf/2: ONE entity,f#2, and a callf(x)resolves to it (the profile note states the fold; adefaults: 1pass-through key says how many arities it covers). Multiple clauses of onef/2are ONE entity anchored from the first clause's line to the last clause'send— the TypeScript overload fold, with theparameterset taken per position across clauses (§16.2). defimplis a named module.defimpl String.Chars, for: MoneydefinesString.Chars.Moneyon the BEAM; the key uses that name in the file where the block is written, andattachedTocarries the relation the name only implies.- Stub keys have the same shape as declared keys — membership is the
whitelist, never the key or the name. Two reserved modules, each a fact
about where the name was NOT found:
<otp>(in the embedded table — the BEAM ships it),<deps>(a module the roots do not declare and the table does not know — a dependency, whichever one).:etsandEnumsit side by side in<otp>: the Erlang/Elixir split is a naming convention on one VM, not a boundary the model should invent. A stub has no members (M15b finding:isStubis a key of TType and TModule only, so a stub function is not representable — the C# fold applies, and a call into an external module lands on its stub module). A local that binds nowhere is dropped and counted, never stubbed.
Pass 1 builds the whitelist: every defmodule (and defprotocol, defimpl)
under the roots, by atom. Then:
| The resolver says | The model says |
|---|---|
| a module under the roots | a declared entity |
| a module in the OTP table | a stub module in <otp>; a function of it, when the table exports that name/arity, a stub function below it; a name the table does not export (a typo, a newer Elixir than the table's) → the stub module, counted |
| a module neither declared nor in the table | a stub module in <deps>; every function called on it a stub function below it, named as written — there is nothing to check it against |
| a local that resolves through the sole non-corpus import | a stub function under that import's <deps> module, counted as import-attributed — what the compiler itself would conclude |
| a local that resolves to nothing | dropped and counted as local-unbound — never a <unresolved> entity for a call; <unresolved> receives only what an edge must target and cannot (an @behaviour/raise naming a module that is neither declared nor known, keyed as written) |
a variable in module position (mod.f()), apply/3, a computed module |
dropped and counted as dynamic-dispatch |
| a file that does not parse | a file record, no entities, counted as unparsed |
Resolvability is not membership — the C# rule, verbatim: Enum resolves and
is external; AcmeOrder.Repo is declared even when use Ecto.Repo bound
nothing.
--deps <dir> (§16.5) is the one bounded place the baseline adds
resolution it otherwise lacks: the dependency sources under deps/*/lib are
parsed for their exports only (names and arities of def/defmacro per
module) so that import Ecto.Query followed by from(u in User) binds to
<deps>/Ecto.Query.from#2 as a fact rather than an attribution, and so that
a __using__ whose body is nothing but import/alias lines can be read
for those lines. Keys do not change — <deps> stays <deps> — only counts do,
every such resolution under its own counter; and a deps/ tree is never
corpus: no entity is emitted from it. The fixture runs without it.
Same flag shape and exit codes as the jar, the C# binary and the TypeScript
bin (schemas/README.md §8). Two additions, both optional:
| Flag | Meaning |
|---|---|
--deps <dir> |
parse dependency sources for their exports only (§16.4); default: none, even when deps/ exists beside the roots — the model of a corpus must not change with what is checked out next to it |
--trace <file> |
merge a compiler trace produced INSIDE the project by mix codegraph.trace --out <file> (a Mix task shipped with the extractor, run by the user where mix compile works): every remote_function/local_function/imported_function/struct_expansion event after macro expansion, keyed by file and position. Edges the baseline already has are unchanged; edges only the trace knows are added with provenance generated; a baseline local-unbound that the trace binds is resolved and the counter says so. Keys never come from the trace — the AST assigns them, the trace only closes edges between them. Shape fixed in M15c after the audit says how much it recovers |
stdout carries nothing but --help/--version; progress and the resolution
summary go to stderr:
✓ walk 412 files (.ex 388, .exs 24) 0.1s
✓ parse 411 parsed, 1 unparsed 1.9s
✓ whitelist 603 modules 0.1s
✓ entities 7 912 entities 0.8s
✓ edges 19 340 edges 1.4s
✓ stubs 541 stubs (<otp> 212, <deps> 318, <unresolved> 11) 0.1s
✓ write 27 806 records 0.3s
RESOLUTION SUMMARY
references : 19 340
resolved : 17 863
unresolved : 1 477
resolution rate : 92.4%
local-unbound (dropped) : 1 102 ← macro-injected imports: the honest ceiling without --trace (profile note)
dynamic-dispatch (dropped): 375
import-attributed : 84
imports : 2 410 (alias 1 630, import 402, require 118, use 260; unresolved: 318)
dynamic-candidate : 96 (protocol 61, GenServer self-calls 35)
entities : 8 453 (stubs: 541)
edges : 19 340 (self-edges dropped: 6)
wrote model.jsonl
- Walking skeleton first: files,
defmodules,def/defpwith arity, the four import forms,@behaviour, stubs — one file in,codegraph validategreen, snapshot committed, the core gatepackages/core/test/fixtures-elixir.test.ts(parses as aModel; byte-identical to core's encoder; ZERO profile issues; closed; no self-edges; the stub-discipline evidence by id). - The fixture corpus
fixtures/elixir/src—acme_orderas a Mix project (mix.exs,lib/,test/,config/) of around 20 files and 400 lines, each pinning one hazard (its README lists them, the TypeScript README's form):application.ex:use Application, achildrenlist →referenceedges to every child module — the supervision tree;order.ex:defstructwith defaults and@enforce_keys, a multi-clausetotal/1,def create(attrs, opts \\ [])— the defaults fold — a pipe (|> Money.add(x)→add#2), a capture,defdelegate,@max_linesread in a body,%__MODULE__{}in a pattern and an update;money.ex:defimpl String.Chars, for: Money→ the named impl module attached toMoney;@derive Jason.Encoder→generated;raise ArgumentError→throwsto<otp>;pricing.ex: a behaviour with two@callbacks and two implementations (@behaviour,@impl true) —interfaceImplementationdeclared;priceable.ex:defprotocolwith twodefimpls andfor: Any→ thedynamic-candidateprotocol call with two candidates;stock.ex:use GenServer,GenServer.call(__MODULE__, {:reserve, id})→ candidates = the threehandle_call/3clauses folded to one, an:ets.lookup/2call →<otp>/ets, aGenServer.call(pid, …)dropped;repo.exandschema/line.ex:use Ecto.Repo,use Ecto.Schema,import Ecto.Changeset,field :qty, :integer(local-unbound, counted),cast(line, attrs, [:qty])(import-attributed to<deps>/Ecto.Changeset),from(l in Line)underimport Ecto.Query— the--depscase in a test, not in the snapshot;macros.ex: a corpusdefmacro __using__used by another module — the__using__#1invocation resolves to a declared macro;notifier.ex: nesteddefmoduleauto-alias,alias AcmeOrder.{Order, Money},alias Money, as: M,import Kernel, except: [length: 1]plus a locallength/1of its own, a function-scopedalias;errors.ex:defexception,raise "text"→<otp>/RuntimeError,reraise;dynamic.ex:apply/3,mod.f(),Module.concat— dropped and counted;spec.ex:@spec/@typenaming remote types →reference;legacy/broken.ex: a file with a syntax error — skipped and counted;two_modules.ex: twodefmodules in one file;promo#2024.ex: a#in a file name;defmodule :"legacy.mod": an atom-named module with a dot;test/acme_order/order_test.exs:use ExUnit.Case,.exswalked like.ex;config/config.exs:import Configand top-levelconfig/3calls FROM the file.
- Extractor-side tests (ExUnit): every line against the per-record schema
for its
t(a JSON Schema validator in test only, never at runtime), the sequence rules, dense surrogates, canonical order;DeterminismTestshuffles the walk order and flips line endings;StubDisciplineTestasserts membership by whitelist on a corpus whose module names mimic OTP's (defmodule Enum.Extrais corpus;Enumis not);ScopeTestandArityTestas tables;stream_dataproperties for the key round trip and for the clause fold (n clauses → one entity spanning all). - Two oracles Elixir alone offers. On a corpus that compiles (the
standard library's
lib/elixir, Phoenix),mix xref graph --format dotis the compiler's module graph: the baseline's file-levelimportlayer, folded to modules, must reproduce its compile-time edges (alias,require,use,import) exactly, and its runtime edges up to the documented blind spots. And on the same corpus the trace (--trace) is the richer extractor of the cross-validation rule: the baseline's edge set must be a subset of the traced one, every gap a named resolution miss in the audit table, never noise. - Every per-fixture suite (analyzer, city, navigator, CLI
validate, corefixtures-*) gains the Elixir fixture; every CLI command is run on it; the city and navigator are reviewed as screenshots — one building perdefmodule, districts by directory (lib/acme_order/schema/), thedynamic-candidatearcs visibly distinct.
- The escript (
bin/codegraph-elixir, built bymix escript.build): runs anywhere Erlang/OTP ≥ 27 is installed; Elixir itself is embedded in the archive. This is the jar: the development and CI form. - The Burrito binary (
extractors/elixir/dist/<rid>/codegraph-elixir): a Mix release plus ERTS in one executable per target, cross-built from one host; needs nothing on the machine. The published artifact — and the M14 sidecar. Both forms mustcmpthe fixture snapshot (the §13.7 gate, twice), and theelixir-smokejob runs the binary on the three OS runners. - A Hex package (
codegraph_elixir) for themix codegraph.tracetask, so a project can produce the trace with one dependency; published on av*tag whenHEX_API_KEYis present (skipped, not failed, otherwise) — thenpm-publishshape.
| Hazard | Where it shows | Guard |
|---|---|---|
| canonical order compares UTF-16 code units; Elixir strings are UTF-8 binaries | any name with a character above U+FFFF or in the surrogate-adjacent range sorts differently under byte order | comparison key = :unicode.characters_to_binary(s, :utf8, {:utf16, :big}), compared as bytes; the fixtures/unicode corpus (the Java/C# guard) reproduced byte for byte |
JSON bytes must match core's JSON.stringify |
escaping (
, control characters, / unescaped), key order, integers vs floats in metrics |
the two-encoder gate in core; a JsonWriterTest with the characters JS and Elixir disagree on by default; JSON.encode_to_iodata! with an explicit string encoder where the defaults differ |
| a source file that is not valid UTF-8 (Latin-1 legacy) | Code.string_to_quoted rejects it; a lone surrogate cannot exist in a binary |
counted as unparsed, path on stderr; --encoding latin1 is a later flag, not a default that guesses |
\r\n sources |
Windows checkouts | the tokenizer is EOL-agnostic; the CRLF determinism case; sloc counted from tokens, not from line splits |
\ path separators and drive letters |
Windows anchors and module keys | Path.relative_to/2 + a forced / join before a path enters a key; the Windows smoke job |
| case-insensitive file systems | enumeration order differs, Foo.ex/foo.ex cannot coexist |
ordinal sort of walked paths by UTF-16 units; the walk never trusts File.ls order |
| the OTP table drifts with the building toolchain | a module built with Elixir 1.19 resolves Enum.new_fn/1, one built with 1.18 counts it |
the table is a build artifact named by version and pinned in the header's extractor keys; the fixture snapshot names the version it was made with and CI builds with exactly it (erlef/setup-beam versions in .tool-versions) |
escript on Windows |
no shebang; escript.exe must be on PATH |
the registry entry names the launcher; the Burrito binary is the Windows deliverable |
| Burrito's cross-build | needs Zig and downloads ERTS per target at build time | build.sh --elixir --native checks for Zig and names the ERTS cache; CI caches it; the escript is the fallback every job has |
an .exs that is a script, not a module |
mix.exs, config/*.exs, seeds.exs call at top level |
the file kind licenses TWithInvocations/TWithAccesses (§16.2); a top-level defmodule in an .exs is a module like any other |
| memory on very large corpora | the whole quoted AST of a file is held during its pass; nothing is held across files but the whitelist and the tables | per-file passes, streaming write; the standard library's lib/elixir (≈ 900 files) is the size gate |
- M15a — profile + skeleton + contract.
feat(core): elixir profile(§16.2, the tenth) and the registry test;extractors/elixir/as a Mix project with the embedded OTP table; the walking skeleton (§16.6 first bullet) →fixtures/elixir/expected/model.jsonlbyte-identical to core's encoder and profile-valid with zero issues; the core gate; the escript onbin/codegraph-elixir;build.sh --elixir/test.sh --elixirwith the escriptcmp;elixir-testin CI. - M15b — the model. Every kind and edge of §16.2 (clause and default
folds, the four import forms,
defimplas an attached named module,@deriveasgenerated,useas import +__using__invocation, struct expansion accesses, protocol andGenServerself-calldynamic-candidates,throws,@specreferences, docs as comments),sloc+cyclomatic, literals; the full fixture with its README; the determinism, stub-discipline, scope and arity suites;--deps; the fixture in every per-fixture suite; every CLI command verified on it; city and navigator screenshots reviewed. - M15c — audit + oracles + distribution. Three corpora, every model
validate-clean:elixir-lang/elixirlib/elixir(the standard library — the size gate and themix xreforacle),phoenixframework/phoenix(macros everywhere: the ceiling measured), an Ecto-heavy application (plausible/analytics: schemas, changesets, queries — the--depsand--tracevalue measured); themix xrefsubset check and the trace superset check as tests with an opt-in real-corpus run (CODEGRAPH_CORPUS_ELIXIR, the M11 pattern);mix codegraph.traceand--traceshaped by what the audit shows; the defects found named in the profile notes with their counts; Burrito binaries,elixir-smokeon three OS runners, the M14 registry entry and cask stanza, the Hex package on a tag;docs/elixir-extractor.md; README and CLAUDE.md name the extractor.
Definition of done: fixtures/elixir/expected/model.jsonl byte-identical to
core's encoder and profile-valid with zero issues; the same bytes from the
escript and from the Burrito binary on Ubuntu, macOS and Windows runners; the
baseline's module layer equal to mix xref's compile-time graph on the
standard library and its edge set a subset of the traced one, with every gap
named; the audit numbers in the profile notes; ./test.sh and CI green with
the Elixir fixture in every per-fixture suite.
| # | Milestone | Definition of done |
|---|---|---|
| M0 | Bootstrap | workspace builds, CI green |
| M1 | Core metamodel | traits + 9 profiles + validation + JSON Schema, tested |
| M2 | Java extractor | ✅ fixture corpus → valid model.json, schema-validated and profile-validated across the language boundary, closed graph, 94.1% resolution on the fixtures |
| M3 | Analyzer | ✅ import graph, type deps, cycles, coupling metrics, DOT/CSV/JSON exports; 252 tests; verified end to end on google/gson (3 624 entities) and apache/commons-lang (15 338 entities) |
| M4 | CLI + properties | ✅ validate/analyze/export/profiles shipped; conformance gate + property suite green; 1 007 TS tests; verified end to end on apache/commons-lang (15 338 entities / 24 631 edges, every command < 1 s) |
| M5 | Metamodel v2 | ✅ METAMODEL.md §1.1/§4 restated + §5.1/§5.2 added + §8 model/encoding split; core natural-key vocabulary, renderId, memoized validation; 1 089 TS tests, wire format and schemas/ byte-identical |
| M6 | JSONL interchange | ✅ in-place clean break (schemaVersion unchanged): streaming reader/writer, per-record schemas + generated container contract, extractor emits .jsonl, fixtures regenerated, v1 deleted; fineract 559.5MB → 127.4MB and analyzable end to end; 1 149 TS + 153 Java tests |
| M7 | SQLite store | codegraph import → model.db cache; DB-backed analyzer facade; repeat runs skip parsing; reports byte-identical to M6 outputs |
| M8 | 2nd language | clj-kondo adapter; cross-language import-graph query works |
| M9a | SCM miner + Gource replay | codegraph scm → deterministic history.jsonl; history summary/hotspots/authors reports match hand-counted fixture numbers; file-level city replay with timeline scrubber runs on codegraph's own history |
| M9b | Temporal store | ✅ sampled import --at revisions in model.db (orchestrated by codegraph snapshots); lifespans + codegraph timeline; hidden-coupling/ownership queries verified on gson history (55 release keyframes, 2008–2025); property suite green at every keyframe |
| M9c | Entity-level city replay | ✅ frozen union layout; temporal city.json with per-building series (codegraph replay); time colors (heat + age), owner color mode and dashed co-change arcs via the --history join; scrubbed replay reviewed as screenshots on gson at user-facing angles, allocation-free scrub path |
| NV | Navigator frontend | ✅ codegraph navigator --serve (since folded into codegraph serve: one page, the navigator with the city as a tab, a building's panel opening its navigator entry): @codegraph/navigator builds an index-addressed browsable artifact (tree + one classified dependency row per base edge, reference sub-roles recovered from the source entity); @codegraph/navigator-ui renders it — virtualized tree with search, fan-in/fan-out sectioned by role with member, provenance and anchor. Design record: docs/navigator.md. Verified on fineract (102 972 nodes / 668 286 rows, 100 MB artifact loading in 1.3 s, 45 rows mounted after scrolling) |
| M10a | Source links | ✅ header repository facts (remote, commit, repo-relative root, provider?) in core + schemas/ as patterns; extractor passthrough flags; CLI-derived normalization carried per snapshot frame; store/city/viz carry them and the details panel links at the scrubbed sha — verified on gson (JsonReader.java#L211-L2005 at b3f4ca2, replay retargeting to ed2b25d, fixture city linkless) |
| M10b | Measures | ✅ TMetrics (open keys, finite values) in core, schemas/ and the store (entity_metric rows, DB_VERSION 3); Java extractor emits sloc (lexer-based) + cyclomatic with a lambda's branches charged to the lambda; fixture CC values hand-counted, sloc ≤ span a property; --height sum:cyclomatic --footprint loc reviewed as screenshots on commons-lang (JavaVersion.get hand-count 33 = emitted 33) |
| M10c | Literal values | ✅ Literal + TWithValue + annotationUse in core, schemas/ (a named $defs/Literal) and the store (JSON columns, DB_VERSION 4); Java extractor carries annotation arguments, constant initializers and element defaults, unevaluated where it cannot fold; closure reaches inside values by ENCODING; fixture asserts the four Audited/MAX_LINES cases; gson and commons-lang diagnose clean |
| M10d | Framework semantics | ✅ data-driven Spring/Jakarta framework profile (5 roles, matching on name + module, stub-tolerant); derived DI dynamic-candidate wiring with @Primary/@Qualifier narrowing that states what it narrowed from; analyze --report wiring + city --framework role channel with its legend; hand-verified on spring-petclinic (a6e81a5: 5 injections → the one corpus impl, 4 → honest empty; HEAD: 6 implicit constructor injections), declared view byte-identical before and after |
| M11 | Insights walk | ✅ codegraph explain: @codegraph/insights (pure) + @codegraph/llm (the one SDK importer); units = operations → types → modules; one SCC-condensed dependency graph (calls, type deps, imports, downward containment) so mutually dependent packages/types/methods are ONE unit, Kahn-layered; context packs with dependency explanations at --depth; Specy-vocabulary blocks validated by Zod and sent as strict JSON Schema; Merkle fingerprints (inputs + dependency fingerprints + missing deps, never explanation text) make re-runs incremental; --dry-run/--max-calls/--scope/--concurrency/--max-scc; journal + sorted side-car <model>.insights.jsonl; cycle suite pinned to the analyzer's cycle report (opt-in real-corpus run via CODEGRAPH_CORPUS_MODEL; Fineract: 53 207 units, 18 package tangles, largest 509). Verified live on the Java fixture with gpt-5.6-luna: 77 units, 68 calls, $0.06 all-in, blocks in the Specy vocabulary (Order → entity with identity, StockGuard.ensure → precondition + error event, com.acme.order → APIs/SPI Ledger); the specy:domain-extract-from-code skill consumes the side-car (heuristics/codegraph.md). Design record: docs/insights.md |
| M12a | C# extractor — skeleton | ✅ extractors/csharp/ (Roslyn 5.9 on .NET 10, no MSBuild, BCL ref pack embedded — §13); csharp profile v2 in core; walking skeleton (namespaces, every type kind, delegates + parameters, doc comments, import/inheritance/implements, stubs) → fixtures/csharp/expected/model.jsonl byte-identical to core's encoder and profile-valid with zero issues; 51 .NET tests (per-line schema + sequence rules, stub discipline, determinism incl. CRLF and walk order, id scheme, CLI) + 13 core gate tests; codegraph validate OK; codegraph snapshots --extractor (jar or binary); extractor CLI contract as schemas/README.md §8; build.sh --csharp [--publish-all] / test.sh --csharp with the published-binary cmp |
| M12b | C# extractor — model | ✅ members (incl. implicit and primary constructors, operators, indexers, events, locals, lambdas, local functions), every profile edge kind incl. annotationUse with written values and throws, extension attachedTo, sloc + cyclomatic; synthesized record members fold to their type; stub discipline (BCL stubs in real namespaces, error types in <unresolved>, unbound receivers referenced by name); snapshot 244 entities / 285 edges, byte-identical to core's encoder; 76 .NET tests + 20 core gate tests; the fixture in the analyzer, city, navigator and CLI suites; every CLI command verified on it |
| M12c | C# extractor — binaries + audit | ✅ five-RID dotnet publish matrix cross-published from one Linux host (287 s; ELF x64/aarch64, Mach-O x64/arm64, PE32+); GitHub Actions gate (verify, java, csharp-test, csharp-publish ×5, csharp-smoke on Ubuntu x64/arm64, macOS arm64/Intel and Windows — each binary must reproduce the snapshot byte for byte — and tagged releases with SHA256SUMS); three-corpus audit: Humanizer 97.3 % / 12 146 entities / 6 s, dotnet/eShop 63.5 % / 7 396 / 12 s, OrchardCore 93.4 % / 88 007 entities / 225 184 edges / 54 s, every model validate-clean; five defects found and fixed (signatures carry type arguments, conversion operators their return type, duplicate parameter names their ordinal, same-keyed declarations across projects kept and re-keyed by file, C# 14 extension blocks); resolution causes measured — the SDK's implicit usings and the ASP.NET Core reference pack now in — and the residue named in the profile notes; eShop city and navigator screenshots reviewed; docs/csharp-extractor.md |
| M13a | TypeScript extractor — skeleton | ✅ extractors/typescript/ (the compiler API as the front end, no build, typescript the only runtime dependency — §14); typescript profile v2; walking skeleton → fixtures/typescript/expected/model.jsonl byte-identical to core's encoder and profile-valid with zero issues; core gate; boundary test; snapshots --extractor runs .js under node; test.sh --ts built-bin cmp |
| M13b | TypeScript extractor — model | ✅ members, every edge kind incl. annotationUse with written arguments and throws, space per entity, declaration merging per file, key escaping, JSX invocations, workspace-package resolution without node_modules, sloc + cyclomatic, literals; the full fixture with its README; determinism and stub-discipline tests; the fixture in every per-fixture suite |
| M13c | TypeScript extractor — self-hosting + audit | ✅ codegraph's own model (24 887 entities / 51 255 edges, 12 s, 96.5 %, zero <unresolved>) validate-clean with the package boundaries, the frontend types-only rule and the three import boundaries recovered as graph queries, city and navigator screenshots reviewed; TypeScript 4.9 compiler (52 341 / 120 127, 97.4 %) / nestjs (29 992 / 32 537, 88.6 %) / excalidraw (43 706 / 62 557, 94.4 %) audit with six defects fixed and the causes in the profile notes; typescript-smoke on three OSes and npm-publish on a tag; docs/typescript-extractor.md |
| M14a | Desktop — the daemon | ✅ codegraph serve --app [--data-dir DIR] [--extractors FILE]: loopback capability URL (/<token>/, one stdout JSON line, 404 outside it, 403 on a foreign Origin), extractor registry as data with extension-census detection (a tie is a 422 question), POST /jobs (202/409/422/404) + jobs/current SSE with replay, --data-dir holding model + model.db + artifacts + recents, extraction skipped on an unchanged tree fingerprint and the build on an unchanged build key, a model.jsonl opens with no extractor, exit on stdin EOF/SIGTERM; the page's empty state in app form (recents, three phases + extractor stderr, the open instruction, the "which extractor?" question, failures); 64 tests incl. the fake extractor over real sockets and the real binary as a child; driven headlessly on the Java fixture and the TypeScript extractor, screenshots reviewed |
| M14b | Desktop — the SEA | ✅ one Node single-executable image per OS, ONE program: dist-sea/codegraph.cjs (CJS, everything but built-ins inlined, 2.9 MB) + both frontends and package.json as assets, folded into a copy of the building Node by scripts/sea-build.mjs (postject, macOS re-signing) → dist-sea/<rid>/codegraph; node:sea reached via process.getBuiltinModule; one FrontendAssets seam with a directory and a keyed source; ./build.sh --ts --sea, test.sh gate, five-runner sea CI job + release artifacts; scripts/sea-smoke.mjs green on linux-x64 (131 MB): --version, analyze ×3 byte-identical to the ESM build, serve --app serving the page from the image and opening the Java fixture |
| M14c | Desktop — the shell | ◐ apps/desktop/ (Tauri 2) written and CI-compiled, the WebGL gate on fineract NOT yet run (needs a Mac): the one sidecar (the SEA) by target triple via scripts/sidecar.mjs, daemon lifecycle (spawn, read the port line, navigate, close stdin + kill), the extractor catalogue + codegraph-discovery (PATH + Homebrew prefixes + settings overrides, 8 tests) writing the registry (missing entries carry install, no path), the daemon's not-installed answer + registry re-read on every request (tested, headless screenshots reviewed), the page's install line with "Check again", File › Open Folder… / Open Recent / Rescan / drag-and-drop each one POST /jobs; no @tauri-apps/* outside apps/desktop (boundary test); remaining: tauri dev screenshots on the Java fixture and fineract, the WebGL frame time |
| M14d | Desktop — signed, in the tap | ◐ pipeline + generators written and tested, no release yet: desktop-bundle job per macOS architecture (sidecar signed by CI with the JIT entitlement trio, tauri build signing + notarizing from secrets, codesign/spctl assertions, ad-hoc bundling without secrets), two DMGs + the raw SEA on the release, scripts/homebrew/render.mjs rendering the cask (app + ONE binary stanza) and the three formulae (codegraph-java/-csharp per architecture from SHA256SUMS, codegraph-typescript on Homebrew's node from the npm tarball) and the release job pushing them to defsquare/homebrew-tap over a deploy key (7 tests, docs/distribution.md); outstanding: the Apple credentials and the deploy key in the repository's secrets, the tap repository, a v* tag, and the clean Apple-silicon and Intel Mac checks (brew install --cask defsquare/tap/codegraph opens the app, a Java folder names the install line, the three extractor formulae put the commands on PATH, each reproducing its fixture snapshot) |
| M15a | Elixir extractor — profile + skeleton | elixir profile in core (the tenth: the module is the file, a defmodule is a module kind carrying TType, arity is identity, no inheritance/embedding — §16.2); extractors/elixir/ as a Mix project on the compiler's parser with the embedded OTP table, no runtime dependency (§16); walking skeleton → fixtures/elixir/expected/model.jsonl byte-identical to core's encoder and profile-valid with zero issues; core gate; escript on bin/codegraph-elixir; build.sh --elixir / test.sh --elixir with the escript cmp; elixir-test in CI |
| M15b | Elixir extractor — model | ✅ (2026-09-16) every kind and edge of §16.2: clause and default folds, the four import forms as one edge kind, defimpl as an attached named module, @derive as generated, use as import + __using__ invocation, struct-expansion accesses, protocol and GenServer self-call dynamic-candidates, throws, @spec references, docs as comments; sloc + cyclomatic, literals; --deps (exports only); the full fixture with its README; determinism, stub-discipline, scope and arity suites; the fixture in every per-fixture suite; city and navigator screenshots reviewed. Snapshot 210 entities / 174 edges; 60 ExUnit tests + 3 properties; Plausible re-run: 16 369 entities / 51 736 edges in 5 s, validate-clean, 64.7 % of 119 972 sites resolved or Kernel, the rest counted by reason (local_injected and local_unbound dominate — the macro ceiling --trace exists for) |
| M15c | Elixir extractor — audit + oracles + distribution | ✅ (2026-09-16) elixir-lang/elixir lib (557 files, 27 061 / 78 564, 17 s, 80.8 %), Phoenix (205, 5 828 / 14 314, 76.7 %) and Plausible (1 256, 16 369 / 43 661, 58.2 %) audited validate-clean, byte-identical to core's encoder, the causes in the profile notes; four defects found and fixed (variadic special forms counted as unbound locals, binary specifiers read as calls, repeated head names re-keying parameters, a corpus use treated as no injection source); --explain-dropped lists every dropped site; mix codegraph.trace (a compilation tracer, JSONL events) + --trace merging generated edges through the closing rules (Phoenix: 16 797 events → 5 895 edges); the mix xref witness check (147/148 on Phoenix) and the trace superset check as opt-in real-corpus tests (CODEGRAPH_CORPUS_ELIXIR); the Burrito binary (Zig 0.16, build.sh --elixir --native, test.sh cmp), elixir-smoke on three OS runners, elixir-native on two, hex-publish on a tag; README, CLAUDE.md and docs/elixir-extractor.md name the extractor. Deferred: the M14 registry entry and cask stanza (M14 is not on main yet), the Windows native binary (needs 7z on the runner) |
| Topic | Decision | Rationale |
|---|---|---|
| Schema lib | Zod v4 (not Malli) | types + runtime validation + JSON Schema export from one source |
| Interchange | JSON (not EDN), schemaVersion-ed |
TS-native; JSON Schema is the polyglot contract |
| Traits/profile equality (open point §2) | required ⊆ traits ⊆ required ∪ optional |
strict equality breaks on TComment; free subset hides extractor bugs |
| Marker traits | TWithInvocations etc. contribute no keys |
edge lists live in edges[], not on entities — keeps entities flat and avoids duplication |
| Java extractor language | Java (Maven) subproject, JSON out | Spoon is a JVM lib; the TS side stays extractor-agnostic |
| Lang ids (M1 review) | frozen as declared, abbreviations kept (clj/js/ts) |
the lang id is the EntityId prefix — renaming one invalidates every id an extractor has emitted |
| Stub containment (M3 review) | a stub type carries TChildOf → its stub module; never a corpus one, and never for primitives or in-corpus phantoms |
the analyzer folds to module level by walking parent and may not parse ids; the alternative — a stub being its own container at every level — put classes and primitives into module graphs (88% of gson's module nodes were not modules) |
Profile space (M1 audit) |
space? declared per kind on the Profile, not only on the Entity |
METAMODEL §1.4's "only meaningful in profiles that declare it" is otherwise unenforceable |
| Trait keys in the published schema (M1 audit) | re-stated as if/then conditionals generated from TRAITS |
Zod refinements do not survive z.toJSONSchema(); without them the contract accepted {traits:["TNamed"]} with no name |
| Identity v2 (fineract audit) | structured key (lang, module, symbol, disambiguator?); rendered id strings are display-only, never stored or compared |
559MB model.json broke Node's 512MB string ceiling; ≈76% of the bytes were repeated id/path strings |
| Interchange v2 | JSONL: surrogate ints for all intra-model refs, header dictionaries for closed vocabularies, file-path table, eof count trailer; in-place clean break, schemaVersion kept at 1.0.0, no old-format reader |
streaming in both directions kills the ceiling; ~6× smaller; extractor bar stays "anything that prints JSON lines"; the format is internal-only — bumping a version nobody consumes buys nothing |
children serialization (v2) |
dropped — derived from parent like every other inverse index |
invariant 4 already forbade serialized inverses; v1 carrying it was an inherited inconsistency (and 12MB on fineract) |
| Closure (M6) | enforced by the ENCODING: a reference is a surrogate, so a dangling one is unwritable and unreadable | the check moves from "report it afterwards" to "it cannot exist"; the extractor drops and counts what it cannot close |
| Executable containment (M6) | method/constructor/lambda declare TWithChildren |
their parameters and locals already carried parent; only one direction was licensed, so the model stated a containment its own profile forbade |
| Sync chunked reader (M6) | readModelFileSync reads 1MB at a time rather than making the CLI async |
the ceiling is one JavaScript string, not synchronous I/O — going async would change every command signature and fix nothing |
| SQLite (v2) | model.db is a derived, disposable analyzer cache built by codegraph import — never the contract, never committed |
queryable/incremental/random access for CLI + future viz without sacrificing diffable fixtures, byte-determinism, or the any-language extractor bar |
| Key separators (M5) | / and # reserved in the key's components; renderId validates and throws |
rendering must be injective, or two distinct keys merge into one entity with no error — the M2 overload collision one level up |
| Module component (M5) | a module names ITSELF, with an empty symbol — not its parent module | the parent form breaks the frozen java:com.acme.order id shape and needs a fabricated java module to place the stub package java:java.util |
| Trait-set interning (v2) | rejected — traits ride inline as int arrays; MM-4's validate-once-per-set is reader-side memoization | set indirection saved ~4MB on a ~90MB file but cost a record type, a dedup pass in every extractor, and lines unreadable in isolation |
| Evolution facts (Phase 8) | separate history.jsonl joined on anchor.file paths — never merged into model.jsonl, no fifth provenance value |
repo-scoped facts with a per-commit lifecycle don't belong in a language-scoped structural contract; the provenance set keeps code facts and history inferences unmixable |
| Lineage over time (Phase 8) | natural-key equality across snapshots; a rename is a death + a birth; anonymous entities untracked | invariant 7 makes temporal identity free for named entities; rename matching is heuristic machinery, deferred until a corpus proves it necessary |
| Replay layout (Phase 8) | one layout over the union of all keys that ever existed, plots frozen; buildings animate in place | shelf packing is chaotic — one insertion reshuffles the city; a readable replay needs positional stability more than land density |
| Snapshot strategy (Phase 8) | sampled keyframes via git worktree; per-commit incremental extraction deferred |
full extraction × thousands of commits is prohibitive, 50–200 frames give the replay effect; noClasspath makes non-compiling historic commits extractable |
| Repository provenance (Phase 9) | facts in the header — remote (normalized https), commit (sha), repo-relative root; host URL templates derived by consumers, provider only when the hostname lies |
a serialized URL freezes one host's scheme into the interchange; a sha is a permalink where a branch moves; anchors are relative to the analyzed root, which sits below the repo root (gson) — without the prefix no anchor projects back |
| Measures (Phase 9) | TMetrics open numeric map; canonical key names documented (METAMODEL §3.8), values validated finite, keys deliberately NOT a closed MM-3 vocabulary |
measures are the extractor-innovation surface — a closed set gates every new measure on a core release; loose top-level keys stay legal (container contract) but uncontractual, and only a trait makes sloc/cyclomatic comparable across extractors |
| Measure storage (M10b) | entity_metric(entity_id, key, value) rows in model.db, not a JSON column; the wire writes the map key-sorted, so reading back ORDER BY key restores it |
the store exists to be QUERIED and a measure is precisely what one aggregates ("cyclomatic by package"); a blob would make the one thing measures are for a full-table JSON scan |
| DI wiring (Phase 9) | derived by the analyzer from declared facts + a framework data table; dynamic-candidate provenance, in-memory only, matched by entity name — never id parsing |
the extractor stays framework-blind; the candidate set needs whole-corpus implementor knowledge only the analyzer holds; Spring dispatch is §1.3's dynamic-candidate definition verbatim |
| Literal values (Phase 9) | tagged-union Literal: numbers as canonical decimal text, enum values as type id + simple name, unfoldable constant expressions kept as unevaluated source text; ids inside values obey closure |
a JSON number loses a Java long; a fabricated enum-member stub is the one thing §6 forbids; dropping an unfoldable expression erases a written fact — degraded honesty over silent loss, the stub discipline applied to values |
| Annotation usage (Phase 9) | dedicated annotationUse edge kind carrying arguments, replacing the plain reference — in-place clean break, fixtures regenerated |
overloading reference would make arguments meaningful on one disguised subset of a kind; consumers cannot select annotation usages today without guessing from the target's kind, which a stub target cannot answer |
| C# corpus loading (M12) | Roslyn syntax + semantic model over a hand-built CSharpCompilation; never MSBuildWorkspace/Build.Locator |
the noClasspath contract: legacy corpora have no restorable project graph; MSBuild needs an installed SDK and breaks single-file publishing; a missing package must be a stub, not a build failure |
| C# unresolved types (M12) | IErrorTypeSymbol → stub in the reserved module csharp:<unresolved>, named as written; BCL/metadata types → stubs in their real namespace |
guessing a namespace from using directives would invent an FQN — the exact fabrication the whitelist rule exists to refuse; the using itself still yields the import edge |
| C# type identity (M12) | symbol carries generic arity via Roslyn MetadataName (Foo1`); signatures use erased FQ metadata names with type parameters as ordinals |
Foo, Foo<T>, Foo<T,U> legally coexist in one namespace — Java-style erasure would merge three declarations into one entity (the M2 overload collision one level up) |
| C# BCL references (M12) | Microsoft.NETCore.App.Ref embedded as resources, loaded via CreateFromImage; never Assembly.Location |
Location is empty inside a single-file bundle, so the tutorial approach binds nothing in the shipped binary while passing under dotnet run |
| C# distribution (M12) | self-contained single-file + ReadyToRun per RID, cross-published from Linux; NativeAOT and trimming deferred | Roslyn is not AOT/trim-clean, and NativeAOT needs each target OS's native toolchain; single-file needs only one runner for all five RIDs |
| Extractor CLI contract (M12) | one flag shape and exit-code set for every extractor, written in schemas/README.md; snapshots --extractor (jar → java -jar, else run directly) |
the Node side must stay language-blind: it orchestrates a process, not a language |
| Cross-OS acceptance (M12) | cmp of the published binary's output against the committed snapshot — run in CI on a runner of each OS since the move to GitHub Actions |
byte-determinism (§6 of the contract) makes "works on macOS/Windows" a one-line check; GitHub's hosted macOS and Windows runners make it a gate rather than a laptop ritual |
| C# signatures carry type arguments (M12c) | System.Func2<!!0,System.String>, not the erased System.Func2; conversion operators append their return type |
C# overloads on type arguments alone (Humanizer) and conversions on the return type alone (OrchardCore, 37 on one type); erasure merged written methods into one key and aborted the extraction |
| Same-keyed declarations across projects (M12c) | every one kept: the first in ordinal file order owns the plain key, each later one is re-keyed #in:<file>, each re-keying named on stderr |
a corpus is not a compilation unit (eShop's ten Program.<Main>$, its per-service Extensions); the JDT "first wins" rule the Java profile documents would erase nine services' DI wiring, and a file is a source fact like a lambda's position |
| Implicit usings (M12c) | Microsoft.NET.Sdk's seven global usings added as a synthetic tree by default; the Web SDK's opt-in (--implicit-usings web); none write an import edge |
they live in the generated obj/ file the extractor skips as build output; without them Task/List<T> were OrchardCore's top unresolved names (62.7 % → 93.4 %); the Web set on a mixed corpus makes names ambiguous (OrchardCore's own StartupBase, 345 times) |
| Reference packs (M12c) | the ASP.NET Core shared framework's pack embedded beside the BCL's when the building SDK has it | ILogger<T>, IServiceCollection, WebApplication topped eShop's and OrchardCore's unresolved lists; a shared framework is not a NuGet package and ships with every SDK |
| C# 14 extension blocks (M12c) | members of extension(T t) { … } are members of the enclosing static class with TAttachedTo → the receiver; the block itself is no entity |
Roslyn models the block as a nameless nested type, which failed validate on an empty name (Humanizer); the block names no type the source can reference |
| TypeScript front end (M13) | the typescript compiler API (ts.createProgram + TypeChecker) used directly; ts-morph, parser-only front ends (tree-sitter, swc, oxc, Babel) and scip-typescript rejected |
it is Roslyn's twin — a compilation plus a semantic model, error-tolerant by default; ts-morph wraps every node and lags releases; a parser has no binder, so every call would be a guess; SCIP carries occurrences, not kinds, provenance or spaces |
| TypeScript corpus loading (M13) | one Program over every source file under the roots; tsconfig read for resolution options only; a bare specifier naming a package declared under the roots resolves by package.json name to its source entry when standard resolution fails; a missing package is a stub module |
the noClasspath contract: no build, no node_modules, no project references; a monorepo's own packages are corpus, not dependencies |
| TypeScript extractor boundary (M13) | runtime dependency typescript only; @codegraph/core a devDependency of its tests; a boundary test scans extractors/typescript/src |
the two-encoder gate ("byte-identical to core's encoder") is a statement only while the encoders are independent; the temptation to import core is greatest in core's own language; a one-dependency package runs under npx anywhere |
| TypeScript module identity (M13) | the module is the file, always; every path segment and non-identifier name entering a key percent-encodes /, #, % (and . in names); declaration merging yields one entity per declaring file, edges land on the declaration owning the referenced member |
/ is a reserved separator and rendering must stay injective for every language; a look-alike separator is not typeable in a CLI or a SQL query; containment is where a thing is written (invariant 5) — a <global> module would state otherwise |
| TypeScript external keys (M13) | stubs keyed by npm package name (nearest package.json), lib types in <lib>, unbound names in <unresolved>, an unresolved specifier a stub module keyed by the specifier; --ignore-node-modules produces the fixture |
keys must not depend on what happens to be installed; resolvability is not membership (the C# rule); a dropped import edge would understate fan-out |
| TypeScript distribution (M13) | an npm package run with npx, typescript external to the bundle; a Node single-executable per OS deferred |
Node ≥ 22 is already the pipeline's floor; a bundle that inlines the compiler loses lib.*.d.ts and binds no standard library — the Assembly.Location trap in its Node form, pinned by a test on the built bin |
| Interface members are type-space (M13c) | a method or property written in an interface body carries space: ["type"]; the profile licenses both spaces on those kinds |
the self-hosting query "frontends depend on model packages for types only" was false with interface members in the value space — viz reads city.roles, a shape, not a value; an access to an interface member is a dependency on the interface's shape and is erased at runtime like the interface |
| What an object literal's parts are (M13c) | members of a literal BOUND to a name (const Ops = {…}, static x = {…}) are entities below the binding; members of an unbound literal (array elements, arguments, return values) are none, and a method written there is keyed positionally like an arrow; a class expression bound to a variable or property IS that binding |
2 260 re-keyed collisions on codegraph's own tests came from [{hash, time}, {hash, time}]; an unbound literal is no entity, so its parts cannot be either, while the functions written inside it are still code |
| Unclosable edges (M13c) | the extractor drops an edge whose endpoint no entity declares, counts it on stderr as unclosable, and never aborts |
schemas/README.md §5: a producer that cannot close a reference drops it and says so; two audited corpora aborted the write on a dangling constructor before this net existed — zero on every corpus is the goal, and a non-zero count names an id-scheme gap |
| Desktop shell (M14) | Tauri 2 as window, menu, dialog, drag-and-drop, extractor discovery, sidecar lifecycle, bundling and signing — no model logic in Rust | the analyzer, city and navigator exist in TypeScript and are tested there; a Rust rewrite of any of it is a second implementation of a contract that already has one |
| Distribution granularity (M14) | the app is ONE install (cask: Codegraph.app + codegraph); every extractor is its OWN install (codegraph-java, codegraph-csharp, codegraph-typescript, codegraph-elixir… as formulae in the same tap); the app bundles no extractor and discovers what is installed |
each extractor carries its own runtime, tens of MB per architecture a user of another language never needs; one cask with all of them grows with every language and re-downloads all on any upgrade; extractors and app version independently and meet on the interchange contract — the reason the extractors were separate processes under one command line in the first place |
| The Node side of the app (M14) | ONE Node single-executable image, ONE program: the CLI, the daemon and both frontends; no extractor inside, no dispatch by invoked name | with TypeScript its own install there is nothing to dispatch; the image is the CLI as built, plus assets, and the §13.5 rule (the Node side never learns a language) is a packaging fact with no exception to explain |
| TypeScript extractor as a formula (M14) | codegraph-typescript = the M13 npm package installed by a Homebrew formula with depends_on "node" (std_npm_args), typescript still external |
it IS one install per extractor without a second 110 MB Node image: ~12 MB on Homebrew's shared node, the shape every Node command has in Homebrew; the libs stay beside the compiler so the §14.7 trap never arises; npx codegraph-typescript and the formula are the same tarball |
| The app's page (M14) | served by the daemon on loopback, same origin, under a per-launch capability token; no Tauri IPC in the page | navigator-ui stays Tauri-free and its boundary test stays simple; CORS between the webview origin and loopback never arises; a code model on an open loopback port is readable by any local page |
| Extractor selection in the app (M14) | a registry { name, path?, extensions[], launch, env?, install? } written by the shell from a catalogue + discovery on PATH and the Homebrew prefixes; detection by extension census; a tie is asked, never guessed; a tree only a missing extractor claims is not-installed with the brew install line |
profiles are data (invariant 8) and so is this: the daemon runs a registry entry under the §13.5 contract and names no language; an absent extractor is a fact the page states with its remedy, not a silent failure |
| Daemon lifetime (M14) | exits on stdin EOF and on SIGTERM; the shell also kills it on close | an orphaned daemon keeps a model in memory and a port open after the window is gone; two rules make that impossible rather than unlikely |
| Homebrew (M14) | one tap, defsquare/homebrew-tap: a cask for the app, one formula per extractor, each generated by its own release job; brew upgrade is the update path, the Tauri updater off; Linux and Windows on the releases, outside brew |
Tauri produces a .app, which a cask installs and a formula cannot; a command on PATH is what a formula installs, from a prebuilt binary that is legitimate in our own tap (homebrew-core refuses vendored binaries, a tap does not); two update mechanisms drift |
| Elixir front end (M15) | the compiler's parser as a library (Code.string_to_quoted/2) plus a lexical resolver and an embedded OTP export table; compilation tracers only as the --trace enrichment; mix xref, tree-sitter-elixir and editor tooling rejected as baselines |
binding in Elixir exists only after macro expansion, which needs every dependency present — a Phoenix corpus with no deps/ fails at its first use; the parser needs nothing, so the noClasspath contract is met by construction and the tracer adds generated facts where a project still compiles |
| Elixir host and distribution (M15) | written in Elixir; an escript (Erlang on the machine, Elixir embedded) and a Burrito binary per OS (nothing on the machine); no runtime dependency (Elixir ≥ 1.18 JSON) |
the quoted AST is the reference form and is reachable only from the BEAM; the escript is the jar and Burrito the native image, so test.sh and the M14 registry keep their shapes |
| Elixir module identity (M15) | the module is the file (the TypeScript rule); a defmodule is a module kind carrying TType, child of its file, nesting an alias not a containment; a module atom's dots are percent-encoded inside the symbol |
Elixir has no namespace to contain a module but the file (invariant 5); the module IS the type (struct, behaviour, protocol), so one building per defmodule is the honest city; the capitalization rule that would make raw dots injective fails for defmodule :"a.b" |
| Elixir function identity (M15) | arity is the disambiguator, always present; f(a, b \\ 1) is one entity f#2 covering f/1; n clauses are one entity spanning first to last; defimpl is the named module the compiler defines, attachedTo the type |
f/1 and f/2 are distinct functions in the language; a default-generated arity and a clause are not declarations of their own (the TypeScript overload fold); the BEAM names the impl module, the model only adds the relation the name implies |
| Elixir external keys (M15) | three reserved modules — <otp> from the embedded table, <deps> for a module nothing declares, <unresolved> for a target an edge must name and cannot; a local that binds nowhere is dropped and counted, never an entity; --deps parses dependency sources for exports only and changes counts, never keys |
keys must not depend on what is checked out beside the corpus (the TypeScript rule); a deps/ tree is not corpus; Erlang and Elixir modules share one VM and one reserved module |
| Elixir dynamic dispatch (M15) | protocol calls and GenServer.call(__MODULE__, …)-style self-calls are dynamic-candidate invocations written by the extractor with the corpus impls / handler clauses as candidates; every other dynamic form (apply/3, mod.f(), a pid) is dropped and counted |
the Go interface-call and Clojure multimethod precedent: the extractor holds the whole corpus and the candidate set is a language fact, unlike Spring DI whose set needs a framework table (M10d); a guess beyond that would be an invented edge |
| Language-defined expansions (M15) | use X is an import plus a declared invocation of X.__using__/1; @derive P is a generated interfaceImplementation; what a used macro injects is absent from the baseline and counted as local-unbound |
both expansions are defined by the language, not by a library, so stating them invents nothing; a router's routes or a schema's fields exist only after expansion, and the Clojure profile's rule (generated where known, absent otherwise) applies verbatim |
| Stub members (M15b) | a stub has no members: a call into an external module is an invocation of the stub module; a field of an external struct is no entity |
isStub is a key of TType and TModule only (isStubEntity), so a stub function would be a full function without an anchor — unrepresentable; the C# fold was already the rule for members of a BCL type |
| Kernel is the language (M15b) | a local that binds to Kernel / Kernel.SpecialForms, and a remote Kernel.f (a string interpolation's to_string/1), is resolved and never an edge, counted as kernel |
length/1, +/2, case are the language: an edge to <otp>/Kernel from every function would be noise that hides real fan-out |
| Injected locals (M15b) | a local that binds nowhere in a module that uses a foreign module is local_injected, and the sole-foreign-import attribution is refused there; a static GenServer.call(Server, …) on any corpus module dispatches to its handler |
the Ecto schema case: field :sku, :string under use Ecto.Schema beside import Ecto.Changeset was attributed to Changeset — a guess; a use is the one place the parser knows it cannot see |
| Module bodies (M15b) | a module body is compile-time code: use, plug, schema … do … end, a for that defines — its calls are edges FROM the module (TWithInvocations/TWithAccesses licensed on module); a module attribute's value owns its own edges |
the TypeScript rule for a file's top level and a class's static block; without the licence the model would state edges its profile forbids, or drop the use invocations that make injection visible |
| The trace (M15c) | mix codegraph.trace writes JSONL events from a compilation tracer (call kind, site, caller module+function, callee); --trace merges them AFTER the parser's pass as generated edges through the same closing rules, a call to an undeclared (injected) function counted rather than invented; the task is reachable from a compiled checkout with ERL_FLAGS=-pa …/ebin, no mix.exs edit |
the compiler is the second reader, not the first: keys stay the parser's, the enrichment only closes edges, and a project that does not compile loses nothing it had. mix compile prunes the code path, so the tracer must be loaded before compiling |
| Kernel special forms (M15c) | matched by name alone; binary specifiers after :: are syntax |
for/try/with/case are variadic — 32 118 "unbound" locals on the standard library were assert, test, for/2 and size/1 in binaries |
| Injection source (M15c) | any use — corpus or foreign — makes the module's unbound locals local_injected, and refuses the sole-import attribution |
a corpus macro's __using__ is as opaque to the parser as a dependency's: the standard library's own tests use ExUnit.Case |
| Native binary entry (M15c) | the Burrito release runs the CLI from Application.start/2 on :init.get_plain_arguments/0 when __BURRITO_BIN_PATH is set, and halts; Burrito itself is a build-only dependency; the escript never starts the application (app: nil) |
Burrito boots through elixir start_cli, which would parse --src as Elixir's own options; halting from the application's start pre-empts it without a runtime dependency on Burrito |
| Elixir oracles (M15) | on a corpus that compiles, mix xref graph must equal the baseline's module-level import layer and the --trace edge set must be a superset of the baseline's; both as opt-in real-corpus tests |
the cross-validation rule ("the richer extractor is the oracle, any gap is a missed case") gets two oracles the language ships for free — the first extractor whose ceiling is measured rather than estimated |