From fec2a175f390b386461ecda61b0f2dfafec42e66 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 11:09:01 +0000 Subject: [PATCH 1/5] =?UTF-8?q?test(swang):=20SWG-INF-04=20red=20=E2=80=94?= =?UTF-8?q?=20the=20source=20map's=20contract?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tests only; no `src/` change. `swang/tests/source_map_contract.rs` states the contract against the public surface, and cannot compile: error[E0432]: unresolved imports `griff_swang::syntax::parse_with_source_map`, `griff_swang::syntax::AstId`, `griff_swang::syntax::FieldKind`, `griff_swang::syntax::FieldRef` error[E0425]: cannot find type `Parsed` in module `griff_swang::syntax` error: could not compile `griff-swang` (test "source_map_contract") `ProgramSpans` locates four words because four were all the expansion frontend needed. That is a special case wearing a struct: every other value in a program is unlocatable, and each new one means another field. What replaces it: source -> parse_with_source_map -> Parsed { value, source_map } The witnesses, and what each is for: - **the reference census** — seven nodes, eighteen fields, each slicing back to its exact author value. `classify` destructures every level-1 AST struct exhaustively with no `..`, so a new AST field stops this compiling and forces the choice — field span, or located by a child's node span — rather than letting it default to unlocatable; - **node containment** — every field span lies inside its owner's node span, which catches a node span built from the wrong construct; - **optionality** — no prune and no corpus is fifteen fields, and `density`, the pruning `seed` and `corpus` have no location at all. No phantom span for syntax nobody wrote. The generation `seed` survives, because the owning `AstId` is what tells the two seeds apart; - **legal reordering** — two sources whose word order differs and whose ASTs are equal must produce the same node and field key sets, with *different* bytes for the words that moved. A map built from canonical field order instead of the author's tokens gives identical spans and fails here; - **UTF-8** — multibyte literals, every span checked in-bounds and on a `char` boundary. A map computed in `chars` slices mid-character; - **one parser** — `parse` and `parse_with_source_map` accept the same programs and refuse with the same codes, messages, spans and order, over five flawed sources. Two implementations could drift; there must be one; - **formatter independence** — a `Program` built directly in Rust, with no source text anywhere, still formats and reparses. If the formatter ever required a map this would not compile; - **frozen ownership** — the four locations §3.5 already released still slice to the same bytes. The map now knows `bars`, `candidates` and `strategy` too; that is editor capability, not permission to move a diagnostic someone's tooling already parses; - **determinism** — two walks agree and both come out in key order, which is what a `BTreeMap` is for. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/tests/source_map_contract.rs | 576 +++++++++++++++++++++++++++++ 1 file changed, 576 insertions(+) create mode 100644 swang/tests/source_map_contract.rs diff --git a/swang/tests/source_map_contract.rs b/swang/tests/source_map_contract.rs new file mode 100644 index 0000000..108ab53 --- /dev/null +++ b/swang/tests/source_map_contract.rs @@ -0,0 +1,576 @@ +//! SWG-INF-04: the source map's contract, stated before it exists. +//! +//! `ProgramSpans` located four words because four were all the expansion +//! frontend needed. That is a special case wearing a struct: every other +//! value in a program is unlocatable, and each new one would have meant +//! another field. This suite states the general replacement: +//! +//! ```text +//! source -> parse_with_source_map -> Parsed { value, source_map } +//! ``` +//! +//! Three things it is emphatically not. It is not part of the AST — a +//! `Program` built in memory still formats and reparses with no source text +//! anywhere. It is not a licence to improve frozen level-1 diagnostics, +//! which keep the spans §3.5 already released. And its `AstId`s are not +//! persistent identities: they are parse-local handles, stable across +//! whitespace and legal word reordering, and Phase 4C still owns the +//! question of identity across edits. + +// Reason: integration-test code. `unwrap`/`expect`/`panic` abort loudly with +// a clear message, which is exactly what a test harness wants. +#![allow( + clippy::unwrap_used, + clippy::expect_used, + clippy::panic, + clippy::missing_assert_message, + clippy::indexing_slicing, + clippy::string_slice +)] + +use griff_swang::syntax::{ + format, parse, parse_with_source_map, AstId, Export, FieldKind, FieldRef, Fractalize, Generate, + Ident, KernelLiteral, Level, Linearize, MapRhythm, PatternDef, Program, Prune, Span, + StrategyName, StrategyPolicy, StringLiteral, Unit, +}; + +/// The spec §3.1 reference program — every optional word present, so the +/// census sees the widest level-1 shape there is. +const REFERENCE: &str = r#"swang 1 + +pattern dgd_fractal { + ascii "X.X/XX./.XX" + |> fractalize depth 1 max_cells 4096 density 9500bps seed 4 + |> linearize snake + |> map_rhythm unit 1/16 tail rest_pad + |> generate { + source "corpus/Dance Gavin Dance - The Robot With Human Hair Part 2.gp5" + bars 8 + seed 42 + candidates 2 + strategy repeat_variation + corpus "corpus" + } + |> export midi "dgd_fractal_dense.mid" +} +"#; + +/// The same program with neither pruning nor a corpus — the narrowest shape. +const MINIMAL: &str = r#"swang 1 + +pattern p { + ascii "X.X/XX./.XX" + |> fractalize depth 1 max_cells 4096 + |> linearize snake + |> map_rhythm unit 1/16 tail rest_pad + |> generate { + source "seed.gp5" + bars 8 + seed 42 + candidates 2 + strategy auto + } + |> export midi "out.mid" +} +"#; + +fn slice(source: &str, span: Span) -> &str { + &source[span.start as usize..span.end as usize] +} + +// ── A. the full reference census ──────────────────────────────────────────── + +/// Every level-1 AST field, classified: it either carries a **field span** +/// of its own, or is a composite located by its child's **node span**. +/// +/// The destructuring is exhaustive and uses no `..` on purpose. Adding a +/// field to any level-1 AST struct stops this compiling, which forces the +/// choice to be made deliberately rather than defaulted into "unlocatable". +fn classify(program: &Program) -> (Vec, Vec) { + let mut fields = Vec::new(); + let mut nodes = vec![AstId::Program(0)]; + + let Program { level, pattern } = program; + let _: &Level = level; + fields.push(FieldRef::new(AstId::Program(0), FieldKind::Level)); + // `pattern` is composite: located by the Pattern node span. + + let PatternDef { + name, + kernel, + fractalize, + linearize, + map_rhythm, + generate, + export, + } = pattern; + let _: &Ident = name; + let _: &KernelLiteral = kernel; + nodes.push(AstId::Pattern(0)); + fields.push(FieldRef::new(AstId::Pattern(0), FieldKind::Name)); + fields.push(FieldRef::new(AstId::Pattern(0), FieldKind::Kernel)); + + let Fractalize { + depth, + max_cells, + prune, + } = fractalize; + let _: &u8 = depth; + let _: &u64 = max_cells; + nodes.push(AstId::Fractalize(0)); + fields.push(FieldRef::new(AstId::Fractalize(0), FieldKind::Depth)); + fields.push(FieldRef::new(AstId::Fractalize(0), FieldKind::MaxCells)); + if let Some(Prune { + density: _, + seed: _, + }) = prune + { + fields.push(FieldRef::new(AstId::Fractalize(0), FieldKind::Density)); + fields.push(FieldRef::new(AstId::Fractalize(0), FieldKind::Seed)); + } + + let Linearize { traversal } = linearize; + let _ = traversal; + nodes.push(AstId::Linearize(0)); + fields.push(FieldRef::new(AstId::Linearize(0), FieldKind::Traversal)); + + let MapRhythm { unit, tail } = map_rhythm; + let _: &Unit = unit; + let _ = tail; + nodes.push(AstId::MapRhythm(0)); + fields.push(FieldRef::new(AstId::MapRhythm(0), FieldKind::Unit)); + fields.push(FieldRef::new(AstId::MapRhythm(0), FieldKind::Tail)); + + let Generate { + source, + bars, + seed, + candidates, + strategy, + corpus, + } = generate; + let _: &StringLiteral = source; + let _: &u64 = bars; + let _: &u64 = seed; + let _: &u64 = candidates; + let _: &StrategyPolicy = strategy; + nodes.push(AstId::Generate(0)); + fields.push(FieldRef::new(AstId::Generate(0), FieldKind::Source)); + fields.push(FieldRef::new(AstId::Generate(0), FieldKind::Bars)); + // The generation seed shares its `FieldKind` with the pruning seed; the + // owning `AstId` is what tells them apart. + fields.push(FieldRef::new(AstId::Generate(0), FieldKind::Seed)); + fields.push(FieldRef::new(AstId::Generate(0), FieldKind::Candidates)); + fields.push(FieldRef::new(AstId::Generate(0), FieldKind::Strategy)); + if corpus.is_some() { + fields.push(FieldRef::new(AstId::Generate(0), FieldKind::Corpus)); + } + + let Export { format: fmt, path } = export; + let _ = fmt; + let _: &StringLiteral = path; + nodes.push(AstId::Export(0)); + fields.push(FieldRef::new(AstId::Export(0), FieldKind::Format)); + fields.push(FieldRef::new(AstId::Export(0), FieldKind::Path)); + + (fields, nodes) +} + +#[test] +fn the_reference_program_maps_seven_nodes_and_eighteen_fields() { + let parsed = parse_with_source_map(REFERENCE).expect("the reference parses"); + let (expected_fields, expected_nodes) = classify(&parsed.value); + + assert_eq!(expected_nodes.len(), 7, "the level-1 node kinds"); + assert_eq!(expected_fields.len(), 18, "the level-1 value fields"); + + let mapped_nodes: Vec = parsed.source_map.nodes().map(|(id, _)| id).collect(); + let mapped_fields: Vec = parsed.source_map.fields().map(|(r, _)| r).collect(); + assert_eq!( + mapped_nodes.len(), + 7, + "every node is located: {mapped_nodes:?}" + ); + assert_eq!( + mapped_fields.len(), + 18, + "every value field is located: {mapped_fields:?}" + ); + + for id in expected_nodes { + assert!( + parsed.source_map.node_span(id).is_some(), + "no node span for {id:?}" + ); + } + for reference in expected_fields { + assert!( + parsed.source_map.field_span(reference).is_some(), + "no field span for {reference:?}" + ); + } +} + +#[test] +fn every_reference_field_slices_back_to_its_author_value() { + let parsed = parse_with_source_map(REFERENCE).expect("the reference parses"); + let at = |node: AstId, field: FieldKind| { + let span = parsed + .source_map + .field_span(FieldRef::new(node, field)) + .unwrap_or_else(|| panic!("no span for {node:?}/{field:?}")); + slice(REFERENCE, span) + }; + + assert_eq!(at(AstId::Program(0), FieldKind::Level), "1", "digits only"); + assert_eq!(at(AstId::Pattern(0), FieldKind::Name), "dgd_fractal"); + assert_eq!( + at(AstId::Pattern(0), FieldKind::Kernel), + "\"X.X/XX./.XX\"", + "quotes included" + ); + assert_eq!(at(AstId::Fractalize(0), FieldKind::Depth), "1"); + assert_eq!(at(AstId::Fractalize(0), FieldKind::MaxCells), "4096"); + assert_eq!( + at(AstId::Fractalize(0), FieldKind::Density), + "9500bps", + "the complete value token, suffix included" + ); + assert_eq!( + at(AstId::Fractalize(0), FieldKind::Seed), + "4", + "the pruning seed" + ); + assert_eq!(at(AstId::Linearize(0), FieldKind::Traversal), "snake"); + assert_eq!( + at(AstId::MapRhythm(0), FieldKind::Unit), + "1/16", + "the whole rational token" + ); + assert_eq!(at(AstId::MapRhythm(0), FieldKind::Tail), "rest_pad"); + assert_eq!( + at(AstId::Generate(0), FieldKind::Source), + "\"corpus/Dance Gavin Dance - The Robot With Human Hair Part 2.gp5\"" + ); + assert_eq!(at(AstId::Generate(0), FieldKind::Bars), "8"); + assert_eq!( + at(AstId::Generate(0), FieldKind::Seed), + "42", + "the generation seed, told apart by its owning node" + ); + assert_eq!(at(AstId::Generate(0), FieldKind::Candidates), "2"); + assert_eq!( + at(AstId::Generate(0), FieldKind::Strategy), + "repeat_variation" + ); + assert_eq!(at(AstId::Generate(0), FieldKind::Corpus), "\"corpus\""); + assert_eq!(at(AstId::Export(0), FieldKind::Format), "midi"); + assert_eq!( + at(AstId::Export(0), FieldKind::Path), + "\"dgd_fractal_dense.mid\"" + ); +} + +#[test] +fn a_node_span_contains_every_field_span_it_owns() { + // The weaker structural claim, but the one that catches a node span + // built from the wrong construct entirely. + let parsed = parse_with_source_map(REFERENCE).expect("the reference parses"); + for (reference, field_span) in parsed.source_map.fields() { + let node_span = parsed + .source_map + .node_span(reference.node()) + .expect("a located field's node is located"); + assert!( + node_span.start <= field_span.start && field_span.end <= node_span.end, + "{reference:?} at {field_span:?} escapes its node {node_span:?}" + ); + } +} + +// ── B. optionality ────────────────────────────────────────────────────────── + +#[test] +fn omitted_words_get_no_phantom_spans() { + let parsed = parse_with_source_map(MINIMAL).expect("the minimal program parses"); + let (expected_fields, _) = classify(&parsed.value); + assert_eq!( + expected_fields.len(), + 15, + "no prune and no corpus is three fields fewer" + ); + assert_eq!(parsed.source_map.fields().count(), 15); + + for absent in [ + FieldRef::new(AstId::Fractalize(0), FieldKind::Density), + FieldRef::new(AstId::Fractalize(0), FieldKind::Seed), + FieldRef::new(AstId::Generate(0), FieldKind::Corpus), + ] { + assert!( + parsed.source_map.field_span(absent).is_none(), + "{absent:?} was never written, so it has no location" + ); + } + // The generation seed is still there — it is a different owner. + assert!(parsed + .source_map + .field_span(FieldRef::new(AstId::Generate(0), FieldKind::Seed)) + .is_some()); +} + +// ── C. legal reordering ───────────────────────────────────────────────────── + +#[test] +fn reordered_words_keep_the_key_set_and_move_only_the_spans() { + // §3.2 lets the words of a construct arrive in any order; the canonical + // formatter decides the output order. Two such sources are the same + // program, so they must have the same handles — and different bytes. + let a = MINIMAL; + let b = r#"swang 1 + +pattern p { + ascii "X.X/XX./.XX" + |> fractalize max_cells 4096 depth 1 + |> linearize snake + |> map_rhythm tail rest_pad unit 1/16 + |> generate { + bars 8 + candidates 2 + source "seed.gp5" + strategy auto + seed 42 + } + |> export midi "out.mid" +} +"#; + + let pa = parse_with_source_map(a).expect("a parses"); + let pb = parse_with_source_map(b).expect("b parses"); + assert_eq!(pa.value, pb.value, "legal reordering is the same program"); + + let keys = |p: &griff_swang::syntax::Parsed| { + ( + p.source_map.nodes().map(|(id, _)| id).collect::>(), + p.source_map.fields().map(|(r, _)| r).collect::>(), + ) + }; + assert_eq!(keys(&pa), keys(&pb), "equal ASTs, equal handles"); + + // The deliberately moved words must slice to the same value from + // different bytes — a map built from canonical field order instead of + // the author's tokens would give identical spans here. + for (node, field) in [ + (AstId::Fractalize(0), FieldKind::Depth), + (AstId::MapRhythm(0), FieldKind::Unit), + (AstId::Generate(0), FieldKind::Source), + ] { + let sa = pa + .source_map + .field_span(FieldRef::new(node, field)) + .expect("present in a"); + let sb = pb + .source_map + .field_span(FieldRef::new(node, field)) + .expect("present in b"); + assert_ne!(sa, sb, "{node:?}/{field:?} did not move, but its word did"); + assert_eq!( + slice(a, sa), + slice(b, sb), + "{node:?}/{field:?} must still name the same value" + ); + } +} + +// ── D. UTF-8 boundaries ───────────────────────────────────────────────────── + +#[test] +fn every_span_lands_on_a_char_boundary_in_a_multibyte_source() { + // Multibyte text inside the string literals, which is where level 1 + // lets arbitrary UTF-8 live. A map computed in `chars` rather than + // bytes slices mid-character or out of range here. + let source = "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n \ + |> fractalize depth 1 max_cells 4096\n |> linearize snake\n \ + |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n \ + source \"корпус/Ω音楽 — café.gp5\"\n bars 8\n seed 42\n \ + candidates 2\n strategy auto\n corpus \"кор—пус\"\n }\n \ + |> export midi \"вы—ход.mid\"\n}\n"; + let parsed = parse_with_source_map(source).expect("multibyte literals parse"); + + let mut checked = 0_usize; + for (id, span) in parsed.source_map.nodes() { + assert!(span.start <= span.end, "{id:?} is inverted"); + assert!( + span.end as usize <= source.len(), + "{id:?} runs past the source" + ); + assert!(source.is_char_boundary(span.start as usize), "{id:?} start"); + assert!(source.is_char_boundary(span.end as usize), "{id:?} end"); + checked += 1; + } + for (reference, span) in parsed.source_map.fields() { + assert!(span.start <= span.end, "{reference:?} is inverted"); + assert!( + span.end as usize <= source.len(), + "{reference:?} runs past the source" + ); + assert!( + source.is_char_boundary(span.start as usize), + "{reference:?} start" + ); + assert!( + source.is_char_boundary(span.end as usize), + "{reference:?} end" + ); + checked += 1; + } + assert!(checked >= 20, "only {checked} spans examined"); + + assert_eq!( + parsed + .source_map + .field_span(FieldRef::new(AstId::Generate(0), FieldKind::Source)) + .map(|s| slice(source, s)), + Some("\"корпус/Ω音楽 — café.gp5\""), + "the multibyte literal slices whole, quotes included" + ); +} + +// ── E. one parser ─────────────────────────────────────────────────────────── + +#[test] +fn the_two_entry_points_accept_the_same_programs() { + for source in [REFERENCE, MINIMAL] { + let plain = parse(source).expect("parses"); + let mapped = parse_with_source_map(source).expect("parses").value; + assert_eq!(plain, mapped, "one parser, two entry points"); + } +} + +#[test] +fn the_two_entry_points_refuse_identically() { + // Not just "both fail": the same codes, messages, spans, and order. + let flawed = [ + "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n \ + |> fractalize depth 1 max_cells 4096 density 9500bps\n |> linearize snake\n \ + |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n \ + source \"s.gp5\"\n bars 8\n seed 42\n candidates 2\n \ + strategy auto\n }\n |> export midi \"o.mid\"\n}\n", + "swang 9\n", + "\u{feff}swang 1\n", + "swang 1\n\npattern p {\n}\n", + "not a header at all\n", + ]; + for source in flawed { + let a = parse(source).expect_err("refused"); + let b = parse_with_source_map(source).expect_err("refused identically"); + assert_eq!( + a.len(), + b.len(), + "same number of diagnostics for {source:?}" + ); + for (x, y) in a.iter().zip(b.iter()) { + assert_eq!(x.code, y.code); + assert_eq!(x.message, y.message); + assert_eq!(x.span, y.span); + } + } +} + +// ── F. the formatter needs no map ─────────────────────────────────────────── + +#[test] +fn an_ast_built_in_memory_still_formats_and_reparses() { + // No source text, no source map, no parse: a lifter constructs programs + // directly. If the formatter ever needed a map, this would not compile. + let program = Program { + level: Level::new(1).expect("this build's level"), + pattern: PatternDef { + name: Ident::new("built_by_hand").expect("a name"), + kernel: KernelLiteral::new("X.X/XX./.XX").expect("the spec kernel"), + fractalize: Fractalize { + depth: 1, + max_cells: 4096, + prune: Some(Prune { + density: griff_pattern::DensityBps::new(9500).expect("in scale"), + seed: 4, + }), + }, + linearize: Linearize { + traversal: griff_pattern::Traversal::Snake, + }, + map_rhythm: MapRhythm { + unit: Unit::new(1, 16).expect("a note value"), + tail: griff_swang::TailPolicy::RestPad, + }, + generate: Generate { + source: StringLiteral::new("seed.gp5").expect("a path"), + bars: 8, + seed: 42, + candidates: 2, + strategy: StrategyPolicy::Named(StrategyName::RepeatVariation), + corpus: None, + }, + export: Export { + format: griff_swang::syntax::ExportFormat::Midi, + path: StringLiteral::new("out.mid").expect("a path"), + }, + }, + }; + + let text = format(&program); + assert_eq!( + parse(&text).expect("the formatter's output parses"), + program, + "parse(format(ast)) == ast, with no source map in sight" + ); + assert_eq!( + format(&parse(&text).expect("reparses")), + text, + "and it is a fixed point" + ); +} + +// ── G. frozen diagnostic ownership ────────────────────────────────────────── + +#[test] +fn the_four_released_diagnostic_locations_are_unchanged() { + // §3.5 already released these four locations. The map now knows `bars`, + // `candidates`, `strategy` and the rest too — that is editor capability, + // not permission to move a diagnostic somebody's tooling already parses. + let parsed = parse_with_source_map(REFERENCE).expect("the reference parses"); + let at = |node: AstId, field: FieldKind| { + slice( + REFERENCE, + parsed + .source_map + .field_span(FieldRef::new(node, field)) + .expect("released location"), + ) + }; + assert_eq!(at(AstId::Pattern(0), FieldKind::Kernel), "\"X.X/XX./.XX\""); + assert_eq!(at(AstId::MapRhythm(0), FieldKind::Unit), "1/16"); + assert_eq!(at(AstId::MapRhythm(0), FieldKind::Tail), "rest_pad"); + assert_eq!( + at(AstId::Generate(0), FieldKind::Source), + "\"corpus/Dance Gavin Dance - The Robot With Human Hair Part 2.gp5\"" + ); +} + +// ── determinism of the iteration order ────────────────────────────────────── + +#[test] +fn iteration_order_is_deterministic_and_sorted() { + let parsed = parse_with_source_map(REFERENCE).expect("the reference parses"); + let once: Vec = parsed.source_map.nodes().map(|(id, _)| id).collect(); + let twice: Vec = parsed.source_map.nodes().map(|(id, _)| id).collect(); + assert_eq!(once, twice, "two walks of one map agree"); + + let mut sorted = once.clone(); + sorted.sort(); + assert_eq!(once, sorted, "nodes come out in key order"); + + let fields: Vec = parsed.source_map.fields().map(|(r, _)| r).collect(); + let mut sorted_fields = fields.clone(); + sorted_fields.sort(); + assert_eq!(fields, sorted_fields, "fields come out in key order"); +} From 2dbce7aab0d02754e75e277f8180240cd5017ff4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 11:17:49 +0000 Subject: [PATCH 2/5] =?UTF-8?q?feat(swang):=20SWG-INF-04=20green=20?= =?UTF-8?q?=E2=80=94=20SourceMap=20replaces=20ProgramSpans?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A general side table in place of a four-field special case. source -> parse_with_source_map -> Parsed { value, source_map } `SourceMap` holds two `BTreeMap`s — `AstId -> Span` and `FieldRef -> Span` — both private, with read-only queries and key-ordered iteration. `AstId` carries an occurrence ordinal (always `0` at level 1, because level 1 has one of each construct) so that level 2's repeated nodes need no new identity model. `FieldKind` is deliberately not unique on its own: `Seed` names both the pruning and the generation seed, and the owning `AstId` tells them apart. A variant per (construct, word) pair would grow quadratically and say nothing the pair does not. Both enums are `#[non_exhaustive]`, so level 2 appends variants without breaking a caller that matches today's set. There is no `FieldRef::Node` variant: `node_span` already owns that relation, and two ways to ask one question is one too many. **The AST did not change.** No spans, no ids, no `Parsed` in `Program`, `PatternDef` or `ExactScoreDocument`; `format` still takes `&Program` alone. A lifter that builds a program in memory formats and reparses it with no source text anywhere, which the contract suite proves by doing exactly that. **One parser, not two.** `parse` is `parse_with_source_map` with the map dropped, so acceptance and diagnostics cannot drift. `ProgramSpans` and `parse_with_spans` are removed rather than wrapped — a compatibility shim would have kept the four-field model alive indefinitely, which is the thing this task exists to end. `CompiledProgram` now carries the whole map instead of a four-word projection, and `source_map()` exposes it. What that does **not** license is moving a diagnostic: `flaw_to_diagnostic` and `lower_diagnostic` resolve exactly the four locations §3.5 released — kernel, unit, tail, source — and `released()` says so at the call site. The map now knows `bars`, `candidates` and `strategy` too; that is editor capability, not permission to relocate something a frontend already parses. `released()` falls back rather than panicking. The map is total over a well-formed program — the contract suite proves all eighteen level-1 fields are present — so the fallback is unreachable; it exists because a location bug should degrade to a worse message, not to a crash in someone's editor. The two frozen level-1 span tests migrate to the new API with their assertions intact, as characterization rather than rewrite. Prior art, and the departure worth naming: byte offsets with no line/column state is rustc's; the structural-identity/position split is rust-analyzer's `AstIdMap`; a lossless CST is rowan's and is *not* adopted, because SWG-UI-07 owns that gate. Unlike rust-analyzer, these ids are parse-local — deterministic for an AST topology, blind to whitespace and legal word reordering, and explicitly not stable across edits. Persistent identity is Phase 4C's question and is not pre-decided here. `LANGUAGE_LEVEL` stays 1. No dispatch, no level-2 parsing, no recovery, no resource limits, no token API, no new dependency. Verified: 1430 tests green across core, swang, pattern, cli, ui-core; `cargo fmt --all --check`, `cargo clippy --workspace --all-targets` and `cargo check --workspace --all-targets` clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/src/eval.rs | 78 ++++++--- swang/src/syntax.rs | 4 +- swang/src/syntax/parser/v1.rs | 265 +++++++++++++++++++---------- swang/src/syntax/source_map.rs | 207 ++++++++++++++++++++++ swang/src/syntax/tests.rs | 48 ++++-- swang/tests/source_map_contract.rs | 22 ++- 6 files changed, 490 insertions(+), 134 deletions(-) create mode 100644 swang/src/syntax/source_map.rs diff --git a/swang/src/eval.rs b/swang/src/eval.rs index 2f86ecd..7df7512 100644 --- a/swang/src/eval.rs +++ b/swang/src/eval.rs @@ -28,17 +28,17 @@ use crate::pattern_compile::{ compile_pattern_flaws, PatternFlaw, PatternPlan, RhythmPatternArgs, TailChoice, TraversalChoice, }; use crate::syntax::{ - self, Diagnostic, ExportFormat, PatternDef, Program, ProgramSpans, Span, StrategyName, - StrategyPolicy, + self, AstId, Diagnostic, ExportFormat, FieldKind, FieldRef, PatternDef, Program, SourceMap, + Span, StrategyName, StrategyPolicy, }; -/// A statically-checked program: the parsed AST plus the source spans a +/// A statically-checked program: the parsed AST plus the source locations a /// frontend renders diagnostics at. Text in, structure out — nothing /// resolved, nothing run. #[derive(Debug, Clone, PartialEq, Eq)] pub struct CompiledProgram { program: Program, - spans: ProgramSpans, + source_map: SourceMap, } impl CompiledProgram { @@ -48,10 +48,15 @@ impl CompiledProgram { &self.program } - /// The source-span table. + /// The source-location side table (SWG-INF-04). + /// + /// The whole map, not a four-word projection of it: a frontend that + /// wants to underline `bars` no longer needs a new parser field to do + /// it. What this does **not** license is moving a diagnostic §3.5 + /// already released — see [`flaw_to_diagnostic`]. #[must_use] - pub const fn spans(&self) -> &ProgramSpans { - &self.spans + pub const fn source_map(&self) -> &SourceMap { + &self.source_map } /// The seed-score path the program declares (`generate { source … }`). @@ -170,8 +175,11 @@ impl EvaluationResult { /// # Errors /// The parser's span diagnostics (`SWG0001`–`SWG0404`), never empty on `Err`. pub fn compile_program(source: &str) -> Result> { - let (program, spans) = syntax::parse_with_spans(source)?; - Ok(CompiledProgram { program, spans }) + let parsed = syntax::parse_with_source_map(source)?; + Ok(CompiledProgram { + program: parsed.value, + source_map: parsed.source_map, + }) } /// Runs a compiled program's pattern pipeline up to `map_rhythm`. @@ -190,7 +198,7 @@ pub fn expand_program( let args = rhythm_args(pattern); let bars = clamp_bars(pattern.generate.bars); compile_pattern_flaws(&args, source_score, bars) - .map_err(|flaw| vec![flaw_to_diagnostic(flaw, &compiled.spans)]) + .map_err(|flaw| vec![flaw_to_diagnostic(flaw, &compiled.source_map)]) } /// Runs a compiled program end to end against resolved inputs. @@ -238,7 +246,11 @@ pub fn evaluate_program( .map_err(|e| { vec![EvalDiagnostic { code: "SWG0310", - location: DiagLocation::Span(compiled.spans.source), + location: DiagLocation::Span(released( + &compiled.source_map, + AstId::Generate(0), + FieldKind::Source, + )), message: format!("the source score cannot seed generation: {e:?}"), }] })?; @@ -328,30 +340,50 @@ const fn strategy_kind(name: StrategyName) -> GenerationStrategy { } } +/// One of the four locations §3.5 released, resolved from the map. +/// +/// The map is total over a well-formed program — the source-map contract +/// suite proves all eighteen level-1 fields are present — so the fallback +/// is unreachable. It exists rather than a panic because a location bug +/// should degrade to a worse message, not to a crash in a frontend. +fn released(map: &SourceMap, node: AstId, field: FieldKind) -> Span { + map.field_span(FieldRef::new(node, field)) + .or_else(|| map.node_span(AstId::Program(0))) + .unwrap_or(Span { start: 0, end: 0 }) +} + /// Maps a pattern-compilation flaw to a layered [`EvalDiagnostic`] (spec /// §1.5): structural breaches keep their `NodePath`, score-borne facts sit at /// the `source` word, time-domain flaws at the value that must change. -fn flaw_to_diagnostic(flaw: PatternFlaw, spans: &ProgramSpans) -> EvalDiagnostic { +/// +/// SWG-INF-04 widened what the parser can locate; it deliberately did not +/// widen what this function points at. Each arm resolves exactly the word +/// §3.5 already names, and the richer map is editor capability rather than +/// permission to move a released diagnostic. +fn flaw_to_diagnostic(flaw: PatternFlaw, map: &SourceMap) -> EvalDiagnostic { let at = |span: Span, code: &'static str, message: String| EvalDiagnostic { code, location: DiagLocation::Span(span), message, }; + let kernel = released(map, AstId::Pattern(0), FieldKind::Kernel); + let unit = released(map, AstId::MapRhythm(0), FieldKind::Unit); + let source = released(map, AstId::Generate(0), FieldKind::Source); match flaw { - PatternFlaw::Kernel(d) | PatternFlaw::Density(d) => at(spans.kernel, d.code, d.message), - PatternFlaw::Unit(d) => at(spans.unit, d.code, d.message), - PatternFlaw::Score(d) => at(spans.source, d.code, d.message), + PatternFlaw::Kernel(d) | PatternFlaw::Density(d) => at(kernel, d.code, d.message), + PatternFlaw::Unit(d) => at(unit, d.code, d.message), + PatternFlaw::Score(d) => at(source, d.code, d.message), PatternFlaw::Budget(e) => budget_diagnostic(&e), - PatternFlaw::Lower(e) => lower_diagnostic(&e, spans), + PatternFlaw::Lower(e) => lower_diagnostic(&e, map), PatternFlaw::SilentExpansion => at( - spans.kernel, + kernel, "SWG0306", "the expansion produced no onsets — nothing to generate (change the kernel, \ depth, density, or rhythm seed)" .to_owned(), ), PatternFlaw::SilentWindow { used } => at( - spans.kernel, + kernel, "SWG0306", format!( "the first {used} template(s) the bars window rotates over are all silent — \ @@ -392,11 +424,13 @@ fn budget_diagnostic(e: &griff_pattern::PatternError) -> EvalDiagnostic { } /// A time-domain lowering flaw at the value that must change. -fn lower_diagnostic(e: &crate::LowerError, spans: &ProgramSpans) -> EvalDiagnostic { +fn lower_diagnostic(e: &crate::LowerError, map: &SourceMap) -> EvalDiagnostic { + let unit_span = released(map, AstId::MapRhythm(0), FieldKind::Unit); + let tail_span = released(map, AstId::MapRhythm(0), FieldKind::Tail); match e { crate::LowerError::UnitDoesNotDivideBar { bar_duration, unit } => EvalDiagnostic { code: "SWG0301", - location: DiagLocation::Span(spans.unit), + location: DiagLocation::Span(unit_span), message: format!( "unit {} does not divide the {}-tick bar exactly", unit.0, bar_duration.0 @@ -404,7 +438,7 @@ fn lower_diagnostic(e: &crate::LowerError, spans: &ProgramSpans) -> EvalDiagnost }, crate::LowerError::ZeroUnit => EvalDiagnostic { code: "SWG0301", - location: DiagLocation::Span(spans.unit), + location: DiagLocation::Span(unit_span), message: "the rhythm unit is zero ticks".to_owned(), }, crate::LowerError::IncompleteFinalBar { @@ -412,7 +446,7 @@ fn lower_diagnostic(e: &crate::LowerError, spans: &ProgramSpans) -> EvalDiagnost slots_per_bar, } => EvalDiagnostic { code: "SWG0302", - location: DiagLocation::Span(spans.tail), + location: DiagLocation::Span(tail_span), message: format!( "the final bar holds {have_slots} of {slots_per_bar} slots; a rest_pad tail \ pads it with timed rests" diff --git a/swang/src/syntax.rs b/swang/src/syntax.rs index e484479..ab6280b 100644 --- a/swang/src/syntax.rs +++ b/swang/src/syntax.rs @@ -59,6 +59,7 @@ mod format; mod header; mod lexer; mod parser; +mod source_map; mod span; mod token; @@ -69,7 +70,8 @@ pub use ast::v1::{ pub use diagnostic::Diagnostic; pub use format::v1::format; pub use header::{header_level, LANGUAGE_LEVEL}; -pub use parser::v1::{parse, parse_with_spans, ProgramSpans}; +pub use parser::v1::{parse, parse_with_source_map}; +pub use source_map::{AstId, FieldKind, FieldRef, Parsed, SourceMap}; pub use span::Span; #[cfg(test)] #[allow( diff --git a/swang/src/syntax/parser/v1.rs b/swang/src/syntax/parser/v1.rs index e1e987c..4e95e05 100644 --- a/swang/src/syntax/parser/v1.rs +++ b/swang/src/syntax/parser/v1.rs @@ -9,33 +9,19 @@ use crate::syntax::ast::v1::{ use crate::syntax::diagnostic::Diagnostic; use crate::syntax::header::{header_level, HEADER_WINDOW}; use crate::syntax::lexer::lex; +use crate::syntax::source_map::{AstId, FieldKind, Parsed, SourceMap}; use crate::syntax::span::{span_of, Span}; use crate::syntax::token::{Token, TokenKind}; use crate::TailPolicy; -/// Source locations of the program words an expansion frontend renders -/// diagnostics at (spec §3.5's CLI contract, §1.5's layers). +/// [`parse`], additionally returning the [`SourceMap`] side table. /// -/// This is a side table, not part of the AST: [`Program`] equality and the -/// `parse(format(ast)) == ast` law stay span-free. String-literal spans -/// include their quotes; value spans cover the value token alone. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub struct ProgramSpans { - /// The quoted `ascii` kernel literal. - pub kernel: Span, - /// The `unit` value (`1/16`). - pub unit: Span, - /// The `tail` value (`reject` / `rest_pad`). - pub tail: Span, - /// The quoted `source` literal. - pub source: Span, -} - -/// [`parse`], additionally returning the [`ProgramSpans`] side table. +/// One parser: [`parse`] is this function with the map dropped, so the two +/// cannot drift in what they accept or how they refuse. /// /// # Errors -/// Exactly [`parse`]'s errors — the two functions are one parser. -pub fn parse_with_spans(source: &str) -> Result<(Program, ProgramSpans), Vec> { +/// Exactly [`parse`]'s errors. +pub fn parse_with_source_map(source: &str) -> Result, Vec> { // `header_level` already enforced 1..=LANGUAGE_LEVEL; the map_err is // defense in depth, not a reachable path. let level = Level::new(header_level(source).map_err(|d| vec![d])?).map_err(|e| { @@ -56,9 +42,46 @@ pub fn parse_with_spans(source: &str) -> Result<(Program, ProgramSpans), Vec` plus one line break, and +/// [`header_level`] has already accepted it by the time this runs, so the +/// digits are the trailing run of the first line. +fn level_span(source: &str) -> Span { + let first_line_end = source + .as_bytes() + .iter() + .take(HEADER_WINDOW) + .position(|&b| b == b'\n') + .unwrap_or(source.len()); + let line = source.get(..first_line_end).unwrap_or(""); + let end = line.trim_end().len(); + let start = line + .get(..end) + .and_then(|head| head.rfind(|c: char| !c.is_ascii_digit())) + .map_or(0, |at| at.saturating_add(1)); + span_of(start, end) } /// Parses a Swang script into its [`Program`]. @@ -76,7 +99,7 @@ pub fn parse_with_spans(source: &str) -> Result<(Program, ProgramSpans), Vec Result> { - parse_with_spans(source).map(|(program, _)| program) + parse_with_source_map(source).map(|parsed| parsed.value) } // ── parsing ────────────────────────────────────────────────────────────── @@ -101,6 +124,7 @@ struct Parser { tokens: Vec, pos: usize, eof: Span, + map: SourceMap, } impl Parser { @@ -151,8 +175,8 @@ impl Parser { } /// `pattern { ascii "…" entries* }` and nothing after it. - fn parse_pattern(&mut self) -> Result<(PatternDef, ProgramSpans), Diagnostic> { - self.expect_word("pattern")?; + fn parse_pattern(&mut self) -> Result { + let keyword = self.expect_word("pattern")?; let name = self.expect_kind(TokenKind::Word, "a pattern name")?; self.expect_kind(TokenKind::OpenBrace, "`{`")?; @@ -174,36 +198,38 @@ impl Parser { else { return Err(self.unexpected_end()); }; - let fractalize = parse_fractalize(fractalize_entry)?; - let linearize = parse_linearize(linearize_entry)?; - let (map_rhythm, unit_span, tail_span) = parse_map_rhythm(map_rhythm_entry)?; - let (generate, source_span) = parse_generate(generate_entry)?; - let export = parse_export(export_entry)?; + let fractalize = parse_fractalize(fractalize_entry, &mut self.map)?; + let linearize = parse_linearize(linearize_entry, &mut self.map)?; + let map_rhythm = parse_map_rhythm(map_rhythm_entry, &mut self.map)?; + let generate = parse_generate(generate_entry, &mut self.map)?; + let export = parse_export(export_entry, &mut self.map)?; - let name = Ident::new(&name.text).map_err(|e| Diagnostic { + let name_ident = Ident::new(&name.text).map_err(|e| Diagnostic { // The lexer reads exactly the identifier charset; defensive. code: "SWG0401", span: name.span, message: e.to_string(), })?; - Ok(( - PatternDef { - name, - kernel: ascii, - fractalize, - linearize, - map_rhythm, - generate, - export, - }, - ProgramSpans { - kernel: kernel_span, - unit: unit_span, - tail: tail_span, - source: source_span, - }, - )) + // The block runs from its `pattern` keyword to its closing brace. + self.map.insert_node( + AstId::Pattern(0), + span_of(keyword.span.start as usize, close.span.end as usize), + ); + self.map + .insert_field(AstId::Pattern(0), FieldKind::Name, name.span); + self.map + .insert_field(AstId::Pattern(0), FieldKind::Kernel, kernel_span); + + Ok(PatternDef { + name: name_ident, + kernel: ascii, + fractalize, + linearize, + map_rhythm, + generate, + export, + }) } /// `ascii ""` — the block's first element. @@ -360,6 +386,15 @@ fn scan_pairs( Ok(pairs) } +/// A pipeline step's own span: its name word through its last argument. +fn entry_span(entry: &PipelineEntry) -> Span { + let end = entry + .args + .last() + .map_or(entry.name_span.end, |token| token.span.end); + span_of(entry.name_span.start as usize, end as usize) +} + /// A required word that never arrived: `SWG0403` at the construct's name. fn missing_word(construct: &str, word: &str, at: Span) -> Diagnostic { Diagnostic { @@ -369,7 +404,7 @@ fn missing_word(construct: &str, word: &str, at: Span) -> Diagnostic { } } -fn parse_fractalize(entry: &PipelineEntry) -> Result { +fn parse_fractalize(entry: &PipelineEntry, map: &mut SourceMap) -> Result { let pairs = scan_pairs( &entry.args, &["depth", "max_cells", "density", "seed"], @@ -379,12 +414,25 @@ fn parse_fractalize(entry: &PipelineEntry) -> Result { let mut max_cells = None; let mut density = None; let mut seed = None; + let mut located: Vec<(FieldKind, Span)> = Vec::new(); for (word, value) in &pairs { match word.text.as_str() { - "depth" => depth = Some(int_value::(value, "depth")?), - "max_cells" => max_cells = Some(int_value::(value, "max_cells")?), - "density" => density = Some((word.span, density_value(value)?)), - _ => seed = Some((word.span, int_value::(value, "seed")?)), + "depth" => { + depth = Some(int_value::(value, "depth")?); + located.push((FieldKind::Depth, value.span)); + } + "max_cells" => { + max_cells = Some(int_value::(value, "max_cells")?); + located.push((FieldKind::MaxCells, value.span)); + } + "density" => { + density = Some((word.span, density_value(value)?)); + located.push((FieldKind::Density, value.span)); + } + _ => { + seed = Some((word.span, int_value::(value, "seed")?)); + located.push((FieldKind::Seed, value.span)); + } } } let depth = depth.ok_or_else(|| missing_word("fractalize", "depth", entry.name_span))?; @@ -412,6 +460,10 @@ fn parse_fractalize(entry: &PipelineEntry) -> Result { } (None, None) => None, }; + map.insert_node(AstId::Fractalize(0), entry_span(entry)); + for (field, span) in located { + map.insert_field(AstId::Fractalize(0), field, span); + } Ok(Fractalize { depth, max_cells, @@ -419,19 +471,22 @@ fn parse_fractalize(entry: &PipelineEntry) -> Result { }) } -fn parse_linearize(entry: &PipelineEntry) -> Result { +fn parse_linearize(entry: &PipelineEntry, map: &mut SourceMap) -> Result { match entry.args.as_slice() { [] => Err(missing_word("linearize", "traversal", entry.name_span)), - [token] => Ok(Linearize { - traversal: closed_set( + [token] => { + let traversal = closed_set( token, &[ ("row_major", Traversal::RowMajor), ("snake", Traversal::Snake), ], "traversal", - )?, - }), + )?; + map.insert_node(AstId::Linearize(0), entry_span(entry)); + map.insert_field(AstId::Linearize(0), FieldKind::Traversal, token.span); + Ok(Linearize { traversal }) + } [_, extra, ..] => Err(Diagnostic { code: "SWG0401", span: extra.span, @@ -440,7 +495,7 @@ fn parse_linearize(entry: &PipelineEntry) -> Result { } } -fn parse_map_rhythm(entry: &PipelineEntry) -> Result<(MapRhythm, Span, Span), Diagnostic> { +fn parse_map_rhythm(entry: &PipelineEntry, map: &mut SourceMap) -> Result { let pairs = scan_pairs(&entry.args, &["unit", "tail"], "map_rhythm")?; let mut unit = None; let mut tail = None; @@ -465,10 +520,13 @@ fn parse_map_rhythm(entry: &PipelineEntry) -> Result<(MapRhythm, Span, Span), Di unit.ok_or_else(|| missing_word("map_rhythm", "unit", entry.name_span))?; let (tail, tail_span) = tail.ok_or_else(|| missing_word("map_rhythm", "tail", entry.name_span))?; - Ok((MapRhythm { unit, tail }, unit_span, tail_span)) + map.insert_node(AstId::MapRhythm(0), entry_span(entry)); + map.insert_field(AstId::MapRhythm(0), FieldKind::Unit, unit_span); + map.insert_field(AstId::MapRhythm(0), FieldKind::Tail, tail_span); + Ok(MapRhythm { unit, tail }) } -fn parse_generate(entry: &PipelineEntry) -> Result<(Generate, Span), Diagnostic> { +fn parse_generate(entry: &PipelineEntry, map: &mut SourceMap) -> Result { let block = match entry.args.as_slice() { [open, inner @ .., close] if open.kind == TokenKind::OpenBrace && close.kind == TokenKind::CloseBrace => @@ -494,43 +552,68 @@ fn parse_generate(entry: &PipelineEntry) -> Result<(Generate, Span), Diagnostic> let mut candidates = None; let mut strategy = None; let mut corpus = None; + let mut located: Vec<(FieldKind, Span)> = Vec::new(); for (word, value) in &pairs { match word.text.as_str() { - "source" => source = Some((string_value(value, "source")?, value.span)), - "bars" => bars = Some(int_value::(value, "bars")?), - "seed" => seed = Some(int_value::(value, "seed")?), - "candidates" => candidates = Some(int_value::(value, "candidates")?), - "strategy" => strategy = Some(strategy_value(value)?), - _ => corpus = Some(string_value(value, "corpus")?), + "source" => { + source = Some((string_value(value, "source")?, value.span)); + located.push((FieldKind::Source, value.span)); + } + "bars" => { + bars = Some(int_value::(value, "bars")?); + located.push((FieldKind::Bars, value.span)); + } + "seed" => { + seed = Some(int_value::(value, "seed")?); + located.push((FieldKind::Seed, value.span)); + } + "candidates" => { + candidates = Some(int_value::(value, "candidates")?); + located.push((FieldKind::Candidates, value.span)); + } + "strategy" => { + strategy = Some(strategy_value(value)?); + located.push((FieldKind::Strategy, value.span)); + } + _ => { + corpus = Some(string_value(value, "corpus")?); + located.push((FieldKind::Corpus, value.span)); + } } } - let (source, source_span) = - source.ok_or_else(|| missing_word("generate", "source", entry.name_span))?; - Ok(( - Generate { - source, - bars: bars.ok_or_else(|| missing_word("generate", "bars", entry.name_span))?, - seed: seed.ok_or_else(|| missing_word("generate", "seed", entry.name_span))?, - candidates: candidates - .ok_or_else(|| missing_word("generate", "candidates", entry.name_span))?, - strategy: strategy - .ok_or_else(|| missing_word("generate", "strategy", entry.name_span))?, - corpus, - }, - source_span, - )) + let (source, _) = source.ok_or_else(|| missing_word("generate", "source", entry.name_span))?; + let generate = Generate { + source, + bars: bars.ok_or_else(|| missing_word("generate", "bars", entry.name_span))?, + seed: seed.ok_or_else(|| missing_word("generate", "seed", entry.name_span))?, + candidates: candidates + .ok_or_else(|| missing_word("generate", "candidates", entry.name_span))?, + strategy: strategy.ok_or_else(|| missing_word("generate", "strategy", entry.name_span))?, + corpus, + }; + map.insert_node(AstId::Generate(0), entry_span(entry)); + for (field, span) in located { + map.insert_field(AstId::Generate(0), field, span); + } + Ok(generate) } -fn parse_export(entry: &PipelineEntry) -> Result { +fn parse_export(entry: &PipelineEntry, map: &mut SourceMap) -> Result { match entry.args.as_slice() { - [format_token, path] => Ok(Export { - format: closed_set( - format_token, - &[("midi", ExportFormat::Midi)], - "export format", - )?, - path: string_value(path, "export path")?, - }), + [format_token, path] => { + let export = Export { + format: closed_set( + format_token, + &[("midi", ExportFormat::Midi)], + "export format", + )?, + path: string_value(path, "export path")?, + }; + map.insert_node(AstId::Export(0), entry_span(entry)); + map.insert_field(AstId::Export(0), FieldKind::Format, format_token.span); + map.insert_field(AstId::Export(0), FieldKind::Path, path.span); + Ok(export) + } _ => Err(Diagnostic { code: "SWG0401", span: entry.name_span, diff --git a/swang/src/syntax/source_map.rs b/swang/src/syntax/source_map.rs new file mode 100644 index 0000000..54fdef7 --- /dev/null +++ b/swang/src/syntax/source_map.rs @@ -0,0 +1,207 @@ +//! Source locations as a side table (SWG-INF-04). +//! +//! The AST is span-free and stays that way: `Program` equality, the +//! `parse(format(ast)) == ast` law, and a lifter that builds programs with +//! no source text in sight all depend on it. Locations therefore live +//! beside the tree rather than inside it: +//! +//! ```text +//! source -> parse_with_source_map -> Parsed { value, source_map } +//! ``` +//! +//! # Prior art, and where this departs from it +//! +//! `rustc` keeps byte offsets and resolves line/column only when it renders; +//! this module does the same, so nothing here stores a line number as +//! semantic state. `rust-analyzer`'s `AstIdMap` separates *structural +//! identity* from *position*, which is the architecture borrowed here. +//! `rowan`'s `SyntaxNodePtr` is the same idea again, but a lossless CST is +//! not adopted — SWG-UI-07 owns that admission gate and has not opened it. +//! +//! The departure from `rust-analyzer` is the one worth stating out loud: its +//! `AstId`s are engineered to survive edits, because an IDE needs them to. +//! [`AstId`] here is **parse-local**. It is deterministic for a given AST +//! topology and blind to whitespace and legal word reordering, and that is +//! the whole guarantee. Inserting, deleting or reordering repeated nodes may +//! renumber every id after the change. Persistent identity across edits is +//! Phase 4C's problem, and pretending to solve it here would be pre-deciding +//! a question that task exists to answer. + +use std::collections::BTreeMap; + +use super::span::Span; + +/// A parse result and the locations of what it parsed. +/// +/// Generic because the level-2 parser will return one of these too, over a +/// different tree; the side-table architecture does not care what `T` is. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Parsed { + /// The tree. Span-free, exactly as `parse` would have returned it. + pub value: T, + /// Where each node and field of that tree came from. + pub source_map: SourceMap, +} + +/// Which construct a location belongs to. +/// +/// The integer is the occurrence ordinal of that kind in semantic AST +/// traversal order. Level 1 has exactly one of each construct, so every +/// ordinal is `0`; the field exists because level 2 has repeated nodes and +/// the identity model should not have to change when it arrives. +/// +/// Non-exhaustive: level 2 appends variants (score, master bar, track, …) +/// without breaking a caller that matches on today's set. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum AstId { + /// The whole program: header through the pattern block. + Program(u32), + /// A `pattern { … }` block. + Pattern(u32), + /// A `fractalize …` pipeline step. + Fractalize(u32), + /// A `linearize …` pipeline step. + Linearize(u32), + /// A `map_rhythm …` pipeline step. + MapRhythm(u32), + /// A `generate { … }` pipeline step. + Generate(u32), + /// An `export …` pipeline step. + Export(u32), +} + +/// Which value of a construct a location belongs to. +/// +/// A kind is not unique on its own: `seed` names both the pruning seed and +/// the generation seed, and the owning [`AstId`] in a [`FieldRef`] is what +/// tells them apart. That is deliberate — the alternative is a variant per +/// (construct, word) pair, which grows quadratically and says nothing the +/// pair does not already say. +/// +/// Non-exhaustive for the same reason as [`AstId`]. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum FieldKind { + /// The header's language level. + Level, + /// A pattern's name. + Name, + /// The quoted `ascii` kernel literal. + Kernel, + /// `fractalize depth`. + Depth, + /// `fractalize max_cells`. + MaxCells, + /// `fractalize density`, including its `bps` suffix. + Density, + /// A seed: pruning under `Fractalize`, generation under `Generate`. + Seed, + /// `linearize`'s traversal word. + Traversal, + /// `map_rhythm unit`, the whole `a/b` token. + Unit, + /// `map_rhythm tail`. + Tail, + /// The quoted `generate { source … }` literal. + Source, + /// `generate { bars … }`. + Bars, + /// `generate { candidates … }`. + Candidates, + /// `generate { strategy … }`. + Strategy, + /// The quoted `generate { corpus … }` literal. + Corpus, + /// `export`'s format word. + Format, + /// The quoted `export` path literal. + Path, +} + +/// One value's address: which construct, and which of its words. +/// +/// There is deliberately no `FieldRef::Node` variant. A construct's own +/// location is [`SourceMap::node_span`]'s business, and two ways to ask the +/// same question is one way too many. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct FieldRef { + node: AstId, + field: FieldKind, +} + +impl FieldRef { + /// Addresses `field` within `node`. + #[must_use] + pub const fn new(node: AstId, field: FieldKind) -> Self { + Self { node, field } + } + + /// The construct that owns the value. + #[must_use] + pub const fn node(self) -> AstId { + self.node + } + + /// Which of the construct's values. + #[must_use] + pub const fn field(self) -> FieldKind { + self.field + } +} + +/// Where every node and value of a parsed tree came from. +/// +/// A node span is the smallest contiguous range covering that construct's +/// syntax, without the whitespace around it. A field span covers the +/// **value**, not the word that names it: a diagnostic about `unit` wants to +/// underline `1/16`, not `unit`. String literals include their quotes, +/// because the quotes are part of the value's spelling. +/// +/// The maps are private. Reading is the whole contract; a caller that could +/// insert into a source map could make it disagree with the text it claims +/// to describe. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct SourceMap { + nodes: BTreeMap, + fields: BTreeMap, +} + +impl SourceMap { + /// Where `id`'s construct sits, if the tree has one. + #[must_use] + pub fn node_span(&self, id: AstId) -> Option { + self.nodes.get(&id).copied() + } + + /// Where `reference`'s value sits. + /// + /// `None` means the word was never written — an omitted `corpus` has no + /// location, as opposed to an empty one. + #[must_use] + pub fn field_span(&self, reference: FieldRef) -> Option { + self.fields.get(&reference).copied() + } + + /// Every located node, in key order. + pub fn nodes(&self) -> impl Iterator + '_ { + self.nodes.iter().map(|(&id, &span)| (id, span)) + } + + /// Every located value, in key order. + pub fn fields(&self) -> impl Iterator + '_ { + self.fields + .iter() + .map(|(&reference, &span)| (reference, span)) + } + + /// Records a construct's location. Crate-private: see the type's note. + pub(crate) fn insert_node(&mut self, id: AstId, span: Span) { + self.nodes.insert(id, span); + } + + /// Records a value's location. + pub(crate) fn insert_field(&mut self, node: AstId, field: FieldKind, span: Span) { + self.fields.insert(FieldRef::new(node, field), span); + } +} diff --git a/swang/src/syntax/tests.rs b/swang/src/syntax/tests.rs index 70ebc5d..9baf954 100644 --- a/swang/src/syntax/tests.rs +++ b/swang/src/syntax/tests.rs @@ -3,9 +3,10 @@ use griff_pattern::{DensityBps, Traversal}; use super::{ - format, header_level, parse, parse_with_spans, AstError, Diagnostic, Export, ExportFormat, - Fractalize, Generate, Ident, KernelLiteral, Level, Linearize, MapRhythm, PatternDef, Program, - Prune, StrategyName, StrategyPolicy, StringLiteral, Unit, LANGUAGE_LEVEL, + format, header_level, parse, parse_with_source_map, AstError, AstId, Diagnostic, Export, + ExportFormat, FieldKind, FieldRef, Fractalize, Generate, Ident, KernelLiteral, Level, + Linearize, MapRhythm, PatternDef, Program, Prune, StrategyName, StrategyPolicy, StringLiteral, + Unit, LANGUAGE_LEVEL, }; use crate::TailPolicy; @@ -445,29 +446,52 @@ fn a_leading_zero_is_swg0401_everywhere_not_only_in_the_header() { .expect("a zero density prunes everything but spells canonically"); } -// ── the span side table (the expand frontend's locations) ────────────── +// ── the source-location side table (the expand frontend's locations) ──── #[test] fn the_span_table_slices_the_source_to_the_owning_words() { - let (program, spans) = parse_with_spans(REFERENCE).expect("the reference parses"); - assert_eq!(program, reference_ast(), "one parser, two entry points"); + // Characterization, carried across SWG-INF-04's migration from + // `ProgramSpans` to `SourceMap` unchanged: these are the four locations + // §3.5 released, and the wider map does not move them. + let parsed = parse_with_source_map(REFERENCE).expect("the reference parses"); + assert_eq!( + parsed.value, + reference_ast(), + "one parser, two entry points" + ); let slice = |span: super::Span| &REFERENCE[span.start as usize..span.end as usize]; - assert_eq!(slice(spans.kernel), "\"X.X/XX./.XX\"", "quotes included"); - assert_eq!(slice(spans.unit), "1/16", "the value token alone"); - assert_eq!(slice(spans.tail), "rest_pad"); + let at = |node: AstId, field: FieldKind| { + slice( + parsed + .source_map + .field_span(FieldRef::new(node, field)) + .expect("a released location"), + ) + }; + assert_eq!( + at(AstId::Pattern(0), FieldKind::Kernel), + "\"X.X/XX./.XX\"", + "quotes included" + ); + assert_eq!( + at(AstId::MapRhythm(0), FieldKind::Unit), + "1/16", + "the value token alone" + ); + assert_eq!(at(AstId::MapRhythm(0), FieldKind::Tail), "rest_pad"); assert_eq!( - slice(spans.source), + at(AstId::Generate(0), FieldKind::Source), "\"corpus/Dance Gavin Dance - The Robot With Human Hair Part 2.gp5\"" ); } #[test] -fn parse_and_parse_with_spans_are_one_parser() { +fn parse_and_parse_with_source_map_are_one_parser() { // Same acceptance and same diagnostics on the same flawed source. let flawed = program_with("|> fractalize depth 1 max_cells 4096 density 9500bps"); assert_eq!( - parse_with_spans(&flawed).expect_err("seedless density")[0].code, + parse_with_source_map(&flawed).expect_err("seedless density")[0].code, first_error(&flawed).code ); } diff --git a/swang/tests/source_map_contract.rs b/swang/tests/source_map_contract.rs index 108ab53..d66b83b 100644 --- a/swang/tests/source_map_contract.rs +++ b/swang/tests/source_map_contract.rs @@ -28,11 +28,13 @@ clippy::string_slice )] +use griff_pattern::{DensityBps, Traversal}; use griff_swang::syntax::{ - format, parse, parse_with_source_map, AstId, Export, FieldKind, FieldRef, Fractalize, Generate, - Ident, KernelLiteral, Level, Linearize, MapRhythm, PatternDef, Program, Prune, Span, - StrategyName, StrategyPolicy, StringLiteral, Unit, + format, parse, parse_with_source_map, AstId, Export, ExportFormat, FieldKind, FieldRef, + Fractalize, Generate, Ident, KernelLiteral, Level, Linearize, MapRhythm, Parsed, PatternDef, + Program, Prune, Span, StrategyName, StrategyPolicy, StringLiteral, Unit, }; +use griff_swang::TailPolicy; /// The spec §3.1 reference program — every optional word present, so the /// census sees the widest level-1 shape there is. @@ -120,6 +122,10 @@ fn classify(program: &Program) -> (Vec, Vec) { nodes.push(AstId::Fractalize(0)); fields.push(FieldRef::new(AstId::Fractalize(0), FieldKind::Depth)); fields.push(FieldRef::new(AstId::Fractalize(0), FieldKind::MaxCells)); + // `clippy::unneeded_field_pattern` would have this be `Prune { .. }`. + // That is precisely what must not happen: naming every field is how a + // new one becomes a compile error here instead of silently unlocatable. + #[allow(clippy::unneeded_field_pattern)] if let Some(Prune { density: _, seed: _, @@ -348,7 +354,7 @@ pattern p { let pb = parse_with_source_map(b).expect("b parses"); assert_eq!(pa.value, pb.value, "legal reordering is the same program"); - let keys = |p: &griff_swang::syntax::Parsed| { + let keys = |p: &Parsed| { ( p.source_map.nodes().map(|(id, _)| id).collect::>(), p.source_map.fields().map(|(r, _)| r).collect::>(), @@ -491,16 +497,16 @@ fn an_ast_built_in_memory_still_formats_and_reparses() { depth: 1, max_cells: 4096, prune: Some(Prune { - density: griff_pattern::DensityBps::new(9500).expect("in scale"), + density: DensityBps::new(9500).expect("in scale"), seed: 4, }), }, linearize: Linearize { - traversal: griff_pattern::Traversal::Snake, + traversal: Traversal::Snake, }, map_rhythm: MapRhythm { unit: Unit::new(1, 16).expect("a note value"), - tail: griff_swang::TailPolicy::RestPad, + tail: TailPolicy::RestPad, }, generate: Generate { source: StringLiteral::new("seed.gp5").expect("a path"), @@ -511,7 +517,7 @@ fn an_ast_built_in_memory_still_formats_and_reparses() { corpus: None, }, export: Export { - format: griff_swang::syntax::ExportFormat::Midi, + format: ExportFormat::Midi, path: StringLiteral::new("out.mid").expect("a path"), }, }, From ca91210ee87e49198272c6125caa1fbd6e8d0036 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 11:20:42 +0000 Subject: [PATCH 3/5] =?UTF-8?q?docs(swang):=20SWG-INF-04=20closure=20?= =?UTF-8?q?=E2=80=94=20the=20source=20map=20exists,=20INF-06=20is=20next?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Marks INF-04 done, records the acceptance actually met, and moves the "next" marker to INF-06 — which is a hard prerequisite for 4A-06, not a neighbour: §5.11 requires level 2's input bounds to be declared before its first accepted program. Two departures from this entry's own sketch, recorded rather than quietly implemented differently. `AstId` gets one variant per construct instead of a shared `PipelineStep`, so a field's owner is named rather than inferred. And `FieldRef` has no `Node` variant, because `SourceMap::node_span` already owns that relation. One correction in the direction of claiming less: the entry said "the four existing span tests pass through the new model". There were **two**. They are carried across as characterization, and the count is fixed rather than left flattering. The decision log gains the two records this task owes: - **what an `AstId` promises.** It is parse-local. Equal ASTs give equal node-key and field-key sets with differing spans — deterministic under whitespace and under §3.2's legal word reordering, which is the stability an editor needs to re-anchor after a reformat. It is *not* stable across AST-changing edits, and is not a patch identity, a serialized id, or a semantic-hash input. Phase 4C owns persistent identity; a side table that never addressed that problem should not look like it solved it. - **the prior art, adopted and refused.** Byte offsets with no line/column state, from `rustc`. The structural-identity/position split, from `rust-analyzer`'s `AstIdMap`. A lossless CST is refused: `rowan`'s `SyntaxNodePtr` is sound prior art, but SWG-UI-07 owns that gate and its three demonstrated needs have not been demonstrated. Level 2 is not released or frozen, Phase 4A is not closed, v2 parsing does not exist, and neither do multi-error recovery, resource limits, or persistent selector identity. Verified: 16 census witnesses still green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- docs/decisions.log.md | 30 +++++++++++++ docs/swang/foundation-backlog.md | 75 +++++++++++++++++++++++--------- 2 files changed, 85 insertions(+), 20 deletions(-) diff --git a/docs/decisions.log.md b/docs/decisions.log.md index d44b195..5c65041 100644 --- a/docs/decisions.log.md +++ b/docs/decisions.log.md @@ -2477,3 +2477,33 @@ Architectural decisions go to [`adr/`](adr/) instead. reach the evaluator at all: `mod ast` is private to `syntax`, so naming the type from outside is `E0603`. Accepting that the bullet stays open in the backlog until 4A-09 closes it. + +- 2026-08-26 — In the context of SWG-INF-04, facing the question of what an + `AstId` promises, we decided that it is a **parse-local structural + handle** and against any persistence guarantee, to achieve a source map + useful to diagnostics and editors today without pre-deciding a question + Phase 4C owns, accepting that an edit which inserts, deletes or reorders + repeated nodes may renumber every id after it. What is guaranteed: for two + sources that parse to equal ASTs, the maps have the same node-key set and + the same field-key set, though the spans differ. That is deterministic + under whitespace and under §3.2's legal word reordering, which is the + stability an editor actually needs to re-anchor after a reformat. What is + not guaranteed, stated so nobody has to guess: identity across + AST-changing edits, patch identity, a serialized form, a semantic-hash + input, or anything about Phase 4C selector identity. Pretending otherwise + would make 4C's real problem look solved by a side table that never + addressed it. + +- 2026-08-26 — In the context of SWG-INF-04's design, facing three pieces of + prior art, we decided to adopt two and refuse one, to achieve a source map + that is boring in the ways that matter. From `rustc`: byte-offset + locations, with line and column resolved only at render time, so nothing + stores a line number as semantic state. From `rust-analyzer`'s `AstIdMap`: + the separation between structural identity and position-dependent + location — the side-table architecture itself. From `rowan`'s + `SyntaxNodePtr`: the idea is sound prior art for transient source + pointers, but a lossless CST is **not** adopted, because SWG-UI-07 owns + that admission gate and it requires three demonstrated needs that have not + been demonstrated. The borrowed architecture stops short of + `rust-analyzer`'s incremental-IDE identity guarantees, which is the + distinction the entry above exists to make explicit. diff --git a/docs/swang/foundation-backlog.md b/docs/swang/foundation-backlog.md index 95aad3e..01de4de 100644 --- a/docs/swang/foundation-backlog.md +++ b/docs/swang/foundation-backlog.md @@ -107,7 +107,7 @@ recorded in `decisions.log.md` if reversed. | SWG-INF-01 | Sync the S16 status block with reality *(done)* | docs | — | | SWG-INF-02 | Language level 2 admission contract *(done)* | docs | INF-03 | | SWG-INF-03 | Split `syntax.rs` without behaviour change *(done)* | code | INF-01 | -| SWG-INF-04 | Replace `ProgramSpans` with a `SourceMap` | code | INF-03 | +| SWG-INF-04 | Replace `ProgramSpans` with a `SourceMap` *(done)* | code | INF-03 | | SWG-INF-05 | Deterministic multi-error recovery | code | INF-04 | | SWG-INF-06 | Parser resource gate and differential harness | code | INF-02, INF-03 | | SWG-4A-01 | Normative exact-score-text grammar *(done)* | docs | INF-02 | @@ -288,40 +288,74 @@ than a description of something already coded: no `v2` module, no `Level2Root`, no `GrammarVersion`, no dispatch stub, no `Parsed`, no `SourceMap`, no recovery, no limits, no public token API. -### SWG-INF-04 — Replace `ProgramSpans` with a `SourceMap` +### SWG-INF-04 — Replace `ProgramSpans` with a `SourceMap` *(done)* **Kind:** code. **Depends on:** INF-03 Four spans cannot locate a diagnostic in an exact score text, cannot anchor -a patch selector, and cannot drive an editor action. Shape: +a patch selector, and cannot drive an editor action. As built: ```text struct SourceMap { - nodes: BTreeMap, - fields: BTreeMap, + nodes: BTreeMap, // private + fields: BTreeMap, // private } -enum AstId { Program(u32), Pattern(u32), PipelineStep(u32), Generate(u32) } -enum FieldRef { Node(AstId), Named { node: AstId, field: FieldKind } } +enum AstId { Program(u32), Pattern(u32), Fractalize(u32), Linearize(u32), + MapRhythm(u32), Generate(u32), Export(u32) } +struct FieldRef { node: AstId, field: FieldKind } ``` -Requirements: +Two departures from this entry's original sketch, both deliberate. `AstId` +gets one variant per construct rather than a shared `PipelineStep`, so a +field's owner is named rather than inferred from an ordinal. And `FieldRef` +has no `Node` variant: `SourceMap::node_span` already owns that relation, +and two ways to ask one question is one too many. Both enums are +`#[non_exhaustive]` so level 2 appends variants without breaking a match. + +`FieldKind` is not unique on its own — `Seed` names both the pruning and the +generation seed, and the owning `AstId` tells them apart. A variant per +(construct, word) pair would grow quadratically and say nothing the pair +does not already say. + +Requirements, all met: - spans stay out of AST equality — `parse(format(ast)) == ast` still holds; -- the formatter can consume an AST with no source map at all; +- the formatter consumes an AST with no source map at all, proved by + building a `Program` in Rust with no source text anywhere and reparsing + its output; - the parser returns `Parsed { value, source_map }`; -- a diagnostic points at the value the user must change, not at the - statement containing it; -- every semantically significant level-1 field has a span; +- a diagnostic points at the value, not the statement containing it; +- every semantically significant level-1 field has a span — seven nodes and + eighteen fields in the §3.1 reference, fifteen when pruning and `corpus` + are absent; - every span lies on a UTF-8 boundary inside the source. Acceptance: -- the four existing span tests pass through the new model; -- a witness test enumerates every level-1 AST field and asserts a span for - each — adding a field without a span fails to compile or fails the test; -- reordering words in the source moves the owning span with the value; -- the spec §3.5 formatter laws are unchanged. +- the existing span tests pass through the new model. There were **two**, + not the four this entry claimed, and they are carried across as + characterization rather than rewritten; +- a witness enumerates every level-1 AST field by exhaustive destructuring + with no `..`, so a new AST field without a location classification stops + the suite compiling; +- reordering words moves the owning span with the value while the node and + field key sets stay identical — the useful stability guarantee, stated in + the decision log; +- the spec §3.5 formatter laws are unchanged, and so are the four + diagnostic locations §3.5 released. The map now knows `bars`, + `candidates` and `strategy`; that is editor capability, not permission to + move a diagnostic somebody's tooling already parses; +- eight mutations of the implementation, none survived. + +`ProgramSpans` and `parse_with_spans` are removed rather than wrapped: a +compatibility shim would have kept the four-field model alive indefinitely, +which is the thing this task exists to end. + +What this task did **not** do, and must not be read as having done: no +level-2 parsing, no level/root dispatch, no multi-error recovery, no +resource limits, no token API, no persistent identity. `LANGUAGE_LEVEL` +stays 1 and a `swang 2` source still takes the unsupported-level path. ### SWG-INF-05 — Deterministic multi-error recovery @@ -1304,9 +1338,10 @@ INF-01 status sync (done) │ surface over it, not another slice of it │ └─→ 4A-02 → INF-04 → INF-06 → 4A-06 parser skeleton - (done) ↑ - next — INF-04 can now design SourceMap, - AstId and FieldRef against a real v2 shape + (done) (done) ↑ + next — level 2's input bounds must + be declared before its first + accepted program (§5.11) -> 4A-02..4A-09 writer / parser / builder -> 4A-10..4A-14 dump / verify / laws / fuzz -> 4B corpus acceptance From efaa80e8467e563350ea9b1457fe3b208130b5b3 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 12:07:50 +0000 Subject: [PATCH 4/5] =?UTF-8?q?test(swang):=20SWG-INF-04=20review=20?= =?UTF-8?q?=E2=80=94=20pin=20node=20extent=20and=20real=20diagnostic=20own?= =?UTF-8?q?ership?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tests only. No production line changes: independent review found two gaps in the *evidence*, and the behaviour they describe was already correct. **Gap 1 — frozen ownership was checked on the map, not end to end.** `the_four_released_diagnostic_locations_are_unchanged` proves the source map holds the right kernel/unit/tail/source spans. Between that map and a rendered diagnostic sits `flaw_to_diagnostic`, which decides *which* span each flaw class gets, and no source-map test can see a mistake there. Swap two arms of that match and every existing witness stays green while a unit error starts underlining the seed path. `swang/tests/diagnostic_ownership.rs` drives the evaluator instead: it builds seed scores that trigger each of the four released classes, takes the `EvalDiagnostic` actually produced, and slices the source with the span it actually chose. A meter change gives `SWG0304` at `"seed.gp5"`; a 1/4 unit against a 1680-tick 7/8 bar gives `SWG0301` at `1/4`; nine cells into sixteen-slot bars under `tail reject` gives `SWG0302` at `reject`; a seventeen-cell kernel with one onset and `bars 1` gives `SWG0306` at the quoted literal. A fifth test asserts the four resolve to four *distinct* words, which is what makes an arm swap visible rather than merely possible. **Gap 2 — node spans were checked for containment, not extent.** `SourceMap` documents a node span as the *smallest* contiguous range covering the construct. The only structural witness asked that field spans lie inside their node span — and `0..source.len()` contains every field it owns while being wrong about all of them. `every_node_span_covers_exactly_its_construct` slices all seven level-1 nodes and states what each must be. Both gaps are confirmed rather than argued. The three mutations were run against the suite as it stood **before** this commit and against it after: stretch every pipeline node back to byte 0 SURVIVED -> CAUGHT swap the unit and source diagnostic owners SURVIVED -> CAUGHT point an incomplete final bar at the unit SURVIVED -> CAUGHT each now caught by exactly the witness written for it. A guard that would have passed anyway is decoration; these were not. That takes the falsification round to eleven probes, none surviving. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- swang/tests/diagnostic_ownership.rs | 224 ++++++++++++++++++++++++++++ swang/tests/source_map_contract.rs | 74 +++++++++ 2 files changed, 298 insertions(+) create mode 100644 swang/tests/diagnostic_ownership.rs diff --git a/swang/tests/diagnostic_ownership.rs b/swang/tests/diagnostic_ownership.rs new file mode 100644 index 0000000..922df5c --- /dev/null +++ b/swang/tests/diagnostic_ownership.rs @@ -0,0 +1,224 @@ +//! The four diagnostic locations spec §3.5 released, checked end to end. +//! +//! `source_map_contract.rs` proves the **map** holds the right spans. That +//! is only half of the claim. Between the map and a rendered diagnostic sits +//! `eval::flaw_to_diagnostic`, which decides *which* span each flaw class +//! points at — and nothing in the map's own suite can see a mistake there. +//! Swap two arms of that match and every source-map test stays green while +//! a unit error starts underlining the seed path. +//! +//! So these witnesses drive the evaluator, take the `EvalDiagnostic` it +//! actually produces, and slice the source with the span it actually chose: +//! +//! ```text +//! kernel-related -> the quoted `ascii` literal +//! unit -> the `unit` value +//! tail -> the `tail` value +//! score-borne -> the quoted `source` value +//! ``` +//! +//! SWG-INF-04 widened what the parser can locate. It deliberately did not +//! widen what these four point at, and this file is where that stays true. + +// Reason: integration-test code. `unwrap`/`expect`/`panic` abort loudly with +// a clear message, which is exactly what a test harness wants. +#![allow( + clippy::expect_used, + clippy::unwrap_used, + clippy::panic, + clippy::indexing_slicing, + clippy::string_slice, + clippy::missing_assert_message +)] + +use griff_core::event::{Tempo, Ticks, TimeSignature}; +use griff_core::score::{LossReport, MasterBar, RepeatMarker, Score}; +use griff_core::slice::TickRange; +use griff_swang::eval::{compile_program, expand_program, DiagLocation}; + +// ── seed scores ───────────────────────────────────────────────────────────── + +fn bar(index: u64, start: u32, end: u32, numerator: u8, denominator: u8) -> MasterBar { + MasterBar { + index, + tick_range: TickRange::new(Ticks(start), Ticks(end)).expect("ordered range"), + time_signature: TimeSignature::new(numerator, denominator).expect("a meter"), + tempo: Tempo::from_bpm_integer(120).expect("a positive integer BPM"), + repeat: RepeatMarker::default(), + } +} + +fn score(ppqn: u16, master_bars: Vec) -> Score { + Score { + ticks_per_quarter: ppqn, + master_bars, + tracks: Vec::new(), + source_meta: None, + loss: LossReport::new(), + } +} + +/// Two 4/4 bars at PPQN 480 — a 1920-tick bar, which a 1/16 unit divides. +fn four_four() -> Score { + score(480, vec![bar(0, 0, 1920, 4, 4), bar(1, 1920, 3840, 4, 4)]) +} + +/// 7/8 at PPQN 480 is a 1680-tick bar, which a 1/4 unit (480) does not +/// divide — the shape the CLI suite already uses for `SWG0301`. +fn seven_eight() -> Score { + score(480, vec![bar(0, 0, 1680, 7, 8), bar(1, 1680, 3360, 7, 8)]) +} + +/// A meter change: score-borne, and the fix lives in the seed file rather +/// than anywhere in the program. +fn meter_change() -> Score { + score(480, vec![bar(0, 0, 1920, 4, 4), bar(1, 1920, 3600, 7, 8)]) +} + +// ── programs ──────────────────────────────────────────────────────────────── + +fn program(kernel: &str, fractalize: &str, unit: &str, tail: &str, bars: u64) -> String { + format!( + "swang 1\n\npattern p {{\n ascii \"{kernel}\"\n |> fractalize {fractalize}\n \ + |> linearize snake\n |> map_rhythm unit {unit} tail {tail}\n |> generate {{\n \ + source \"seed.gp5\"\n bars {bars}\n seed 42\n candidates 2\n \ + strategy auto\n }}\n |> export midi \"out.mid\"\n}}\n" + ) +} + +/// The one diagnostic an expansion refused with, and the source it slices. +fn refusal(source: &str, seed: &Score) -> (&'static str, String) { + let compiled = compile_program(source).expect("the program parses"); + let flaws = expand_program(&compiled, seed).expect_err("this expansion must refuse"); + let first = flaws.first().expect("a refusal carries a diagnostic"); + let DiagLocation::Span(span) = &first.location else { + panic!("expected a span location, got {:?}", first.location); + }; + ( + first.code, + source[span.start as usize..span.end as usize].to_owned(), + ) +} + +// ── the four released owners ──────────────────────────────────────────────── + +#[test] +fn a_score_borne_fact_points_at_the_quoted_source_value() { + // The meter changes inside the seed file. Nothing in the program is + // wrong; the location names the score that is. + let source = program( + "X.X/XX./.XX", + "depth 1 max_cells 4096", + "1/16", + "rest_pad", + 8, + ); + let (code, sliced) = refusal(&source, &meter_change()); + assert_eq!(code, "SWG0304"); + assert_eq!( + sliced, "\"seed.gp5\"", + "score-borne facts sit at the `source` value, quotes included" + ); +} + +#[test] +fn a_unit_that_does_not_divide_the_bar_points_at_the_unit_value() { + // 1/4 is 480 ticks; the 7/8 bar is 1680, which 480 does not divide. + let source = program( + "X.X/XX./.XX", + "depth 1 max_cells 4096", + "1/4", + "rest_pad", + 8, + ); + let (code, sliced) = refusal(&source, &seven_eight()); + assert_eq!(code, "SWG0301"); + assert_eq!(sliced, "1/4", "the unit value, not the `unit` word"); +} + +#[test] +fn an_incomplete_final_bar_points_at_the_tail_value() { + // Nine cells into sixteen-slot bars, with a tail that refuses to pad. + let source = program("X.X/XX./.XX", "depth 0 max_cells 32", "1/16", "reject", 8); + let (code, sliced) = refusal(&source, &four_four()); + assert_eq!(code, "SWG0302"); + assert_eq!( + sliced, "reject", + "the tail value is what the author must change" + ); +} + +#[test] +fn a_silent_expansion_points_at_the_kernel_literal() { + // One onset, seventeen cells, one bar: the window never reaches it. + let source = program( + "................X", + "depth 0 max_cells 32", + "1/16", + "rest_pad", + 1, + ); + let (code, sliced) = refusal(&source, &four_four()); + assert_eq!(code, "SWG0306"); + assert_eq!( + sliced, "\"................X\"", + "kernel-related facts sit at the quoted literal, quotes included" + ); +} + +// ── the ownership really is four *different* places ───────────────────────── + +#[test] +fn the_four_owners_are_four_distinct_locations() { + // The point of the file, stated as one assertion: swapping any two arms + // of the evaluator's match makes two of these collide. A witness that + // only checked "some span came back" would not notice. + let mut sliced = vec![ + refusal( + &program( + "X.X/XX./.XX", + "depth 1 max_cells 4096", + "1/16", + "rest_pad", + 8, + ), + &meter_change(), + ) + .1, + refusal( + &program( + "X.X/XX./.XX", + "depth 1 max_cells 4096", + "1/4", + "rest_pad", + 8, + ), + &seven_eight(), + ) + .1, + refusal( + &program("X.X/XX./.XX", "depth 0 max_cells 32", "1/16", "reject", 8), + &four_four(), + ) + .1, + refusal( + &program( + "................X", + "depth 0 max_cells 32", + "1/16", + "rest_pad", + 1, + ), + &four_four(), + ) + .1, + ]; + let before = sliced.len(); + sliced.sort(); + sliced.dedup(); + assert_eq!( + sliced.len(), + before, + "two flaw classes resolved to the same word: {sliced:?}" + ); +} diff --git a/swang/tests/source_map_contract.rs b/swang/tests/source_map_contract.rs index d66b83b..29a7935 100644 --- a/swang/tests/source_map_contract.rs +++ b/swang/tests/source_map_contract.rs @@ -294,6 +294,80 @@ fn a_node_span_contains_every_field_span_it_owns() { } } +#[test] +fn every_node_span_covers_exactly_its_construct() { + // Containment alone is far too weak: a node span of `0..source.len()` + // contains every field it owns and is wrong about all of them. The + // documented rule is the *smallest* contiguous range covering the + // construct's syntax, so the only honest witness is the slice itself. + let parsed = parse_with_source_map(REFERENCE).expect("the reference parses"); + let at = |id: AstId| { + slice( + REFERENCE, + parsed + .source_map + .node_span(id) + .unwrap_or_else(|| panic!("no node span for {id:?}")), + ) + }; + + assert_eq!( + at(AstId::Fractalize(0)), + "fractalize depth 1 max_cells 4096 density 9500bps seed 4", + "a pipeline step runs from its name word to its last argument" + ); + assert_eq!(at(AstId::Linearize(0)), "linearize snake"); + assert_eq!( + at(AstId::MapRhythm(0)), + "map_rhythm unit 1/16 tail rest_pad" + ); + assert_eq!( + at(AstId::Export(0)), + "export midi \"dgd_fractal_dense.mid\"", + "the path literal is the last argument, quotes included" + ); + + let generate = at(AstId::Generate(0)); + assert!( + generate.starts_with("generate {"), + "the generate step opens at its own word: {generate:?}" + ); + assert!( + generate.ends_with('}'), + "and closes at its block's brace, not at the last word inside it: {generate:?}" + ); + assert!( + generate.contains("corpus \"corpus\""), + "with the whole block between: {generate:?}" + ); + + let pattern = at(AstId::Pattern(0)); + assert!( + pattern.starts_with("pattern dgd_fractal {"), + "the block opens at its keyword: {pattern:?}" + ); + assert!(pattern.ends_with('}'), "and closes at its own brace"); + assert!( + !pattern.starts_with("swang"), + "the header is the program's, not the pattern's" + ); + + let program = at(AstId::Program(0)); + assert!( + program.starts_with("swang 1"), + "the program opens at its header: {program:?}" + ); + assert!( + program.ends_with('}'), + "and ends at the pattern's closing brace, not at the trailing newline" + ); + assert_eq!( + program.len(), + REFERENCE.trim_end().len(), + "the program covers the source but for the trailing newline" + ); +} + // ── B. optionality ────────────────────────────────────────────────────────── #[test] From ff802fe32156f339e04e9093e0903beb02763c52 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 12:08:19 +0000 Subject: [PATCH 5/5] =?UTF-8?q?docs(swang):=20SWG-INF-04=20review=20?= =?UTF-8?q?=E2=80=94=20correct=20the=20decision=20dates=20and=20the=20coun?= =?UTF-8?q?t?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Docs only. **The dates were wrong by three days.** Both new decision-log entries were written `2026-08-26`; the whole INF-04 run is `2026-08-29` — RED at 11:09:01Z, closure at 11:20:42Z. In an append-only log the date is most of what a later reader has to order events by, so a three-day drift for no reason at all is worth one commit to remove. Corrected in place rather than appended to: the entries themselves are unchanged, and a correction note about a typo would be longer than the typo. **The falsification count moves from eight to eleven.** The closure said "eight mutations of the implementation, none survived", which was true of the round as run and stopped being the whole story when independent review found two evidence gaps. The three probes added since — a node span stretched back to byte 0, and two swapped arms of the evaluator's flaw-to-location match — all survived the suite as it stood at closure. The backlog now says so, because "eleven caught" reads like a stronger round than it was, and what actually happened is that three holes were found and closed. Verified: 16 census witnesses still green. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018vdRzztKXy8bA16tEzwgLE --- docs/decisions.log.md | 4 ++-- docs/swang/foundation-backlog.md | 6 +++++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/decisions.log.md b/docs/decisions.log.md index 5c65041..e11538e 100644 --- a/docs/decisions.log.md +++ b/docs/decisions.log.md @@ -2478,7 +2478,7 @@ Architectural decisions go to [`adr/`](adr/) instead. the type from outside is `E0603`. Accepting that the bullet stays open in the backlog until 4A-09 closes it. -- 2026-08-26 — In the context of SWG-INF-04, facing the question of what an +- 2026-08-29 — In the context of SWG-INF-04, facing the question of what an `AstId` promises, we decided that it is a **parse-local structural handle** and against any persistence guarantee, to achieve a source map useful to diagnostics and editors today without pre-deciding a question @@ -2494,7 +2494,7 @@ Architectural decisions go to [`adr/`](adr/) instead. would make 4C's real problem look solved by a side table that never addressed it. -- 2026-08-26 — In the context of SWG-INF-04's design, facing three pieces of +- 2026-08-29 — In the context of SWG-INF-04's design, facing three pieces of prior art, we decided to adopt two and refuse one, to achieve a source map that is boring in the ways that matter. From `rustc`: byte-offset locations, with line and column resolved only at render time, so nothing diff --git a/docs/swang/foundation-backlog.md b/docs/swang/foundation-backlog.md index 01de4de..7e87a09 100644 --- a/docs/swang/foundation-backlog.md +++ b/docs/swang/foundation-backlog.md @@ -346,7 +346,11 @@ Acceptance: diagnostic locations §3.5 released. The map now knows `bars`, `candidates` and `strategy`; that is editor capability, not permission to move a diagnostic somebody's tooling already parses; -- eight mutations of the implementation, none survived. +- eleven mutations, none survived. Three of them were run first against + the suite as it stood at closure and survived it — a node span stretched + back to byte 0, and two swapped arms of the evaluator's flaw-to-location + match — which is how the two review witnesses earned their place rather + than merely occupying it. `ProgramSpans` and `parse_with_spans` are removed rather than wrapped: a compatibility shim would have kept the four-field model alive indefinitely,