diff --git a/docs/decisions.log.md b/docs/decisions.log.md index e11538e..8e6b082 100644 --- a/docs/decisions.log.md +++ b/docs/decisions.log.md @@ -2507,3 +2507,229 @@ Architectural decisions go to [`adr/`](adr/) instead. been demonstrated. The borrowed architecture stops short of `rust-analyzer`'s incremental-IDE identity guarantees, which is the distinction the entry above exists to make explicit. + +- 2026-08-29 — In the context of SWG-INF-06, facing an entry that asked for + the pre-refactor parser to be diffed against the refactored one, we + decided to **retire that comparison as already discharged and record a + frozen Law A baseline in its place**, to achieve a differential with a + living right-hand side, accepting that no finite corpus is Law A's whole + domain. INF-03 deleted the parser the entry wanted to diff against and its + own acceptance already made that comparison — byte-identical reference, + identical test counts, no edited expected value, its own mutation round — + so there is nothing left in the tree to compare and resurrecting dead code + to diff it would be theatre. Spec §5.5 states the differential that will + have a second side: a build supporting `1..=N` treats every source whose + first line is a valid `swang 1` header, invalid *bodies* included, exactly + as a level-1-only build did, on all seven observables. Today N is 1, so the left-hand side is recorded now, + while a level-1-only build is what the tree holds — by the time 4A-06 + supplies the right-hand side, that build will be gone exactly as the + pre-refactor parser is gone now. + +- 2026-08-29 — In the context of the Law A baseline's shape, facing the + question of what makes an AST observation *independent* evidence, we + decided on a **test-owned exhaustive projection**, and against `Debug`, + against serde, and against recording `format(ast)`, to achieve two + witnesses rather than one wearing two hats, accepting that the projection + must be extended by hand whenever the AST grows. `Debug` output is a + representation detail no contract pins; `Program` is deliberately not + serialized, so adding serde would mean inventing a contract in order to + test it; and recording the formatter's output as the AST observation would + leave canonical bytes and the AST as the same measurement, so a + coordinated parser-and-formatter regression could preserve the bytes while + changing what the tree means. Every struct is destructured with no `..` + and every enum matched with no wildcard, so growth is a compile error + rather than a silent blind spot. The artifact is compare-only: no + snapshot-update path exists, because a baseline that regenerates itself + records whatever the code now does and calls it history. + +- 2026-08-29 — In the context of SWG-INF-06's four bounds, facing two axes + that today's grammar cannot approach, we decided to **declare all four and + label depth and diagnostics as forward reservations**, to achieve the one + thing §5.11 leaves no second chance at, accepting that two declared + numbers guard nothing yet. The exact-score grammar has no recursive + production and the parser returns one diagnostic, so 64 and 256 are + currently unreachable; but a bound not declared before level 2's first + accepted program can never be added, because adding it afterwards would + narrow an acceptance set that is by then frozen. Declaring them costs + nothing today and omitting them spends the option permanently. The spec + says plainly that they are reservations and not evidence of a + stack-overflow hazard, because a limit presented as a defence against a + danger that does not exist is how a number stops being questioned. + *(Corrected on review: the reason a later bound is forbidden is §5.11's own + admission rule, which is stricter than the freeze boundary — not that the + level is frozen by then. §5.3 keeps level 2 provisional until Phase 4A is + accepted, so a bound added after the first accepted program would still + predate the freeze. The deadline stands; the causality was wrong.)* + +- 2026-08-29 — In the context of the budget having no level-2 parser to + guard, facing the paradox that INF-06 must precede level 2's first + accepted program while 4A-06 owns the parser, we decided that **INF-06 + ships the mechanism and 4A-06 ships the first live wiring**, recorded as + an inherited bullet in 4A-06's acceptance, to achieve an explicit temporal + boundary instead of a fake gate, accepting that the budget has no caller + until then. Wiring a gate into a parser that does not exist would be the + fake half of the work, and a fuzz oracle asserting a limit breach would + cover an execution path the binary cannot enter. The same reasoning splits + the fuzz work: the registry-shape oracle tightens here, because it is + level-agnostic and was genuinely weak — `starts_with("SWG")` accepted + `SWG`, `SWGxyz`, and `SWG12345` — while the end-to-end breach oracle waits + for a parser a fuzzed input can reach. + +- 2026-08-29 — In the context of the Law A corpus, facing a falsification + probe that survived, we decided to **grow the corpus until it reaches the + production sites rather than merely the codes**, and to pin its extent, to + achieve a sample whose claim matches its evidence, accepting that the + sample is still a sample. Corrupting `SWG0403` at one of its four + production sites survived a corpus that reached every level-1 diagnostic + code, because reaching a code says nothing about the other places that + raise it — `SWG0401` is raised from twenty-two. The corpus grew from 23 + cases and 14 distinct refusals to 50 and 38, and a test now records that + number so a shrinking corpus fails rather than quietly testing less. The + failure was the recurring one: a check too narrow for the data it runs + over, counting codes where the regressions live at sites. + *(Corrected on later review: 50 cases / 38 refusals was the pre-domain-fix + extent. Removing the three out-of-domain header cases left the accepted + Law A corpus at 47 cases / 35 distinct refusals, which is what the test + now pins.)* + +- 2026-08-29 — In the context of SWG-INF-06's review, facing a Law A + baseline that recorded three sources the frozen pre-parser refuses, we + decided that **Law A's domain is itself a witness**, and against keeping + header refusals in the artifact, to achieve a baseline that 4A-06 can + actually satisfy, accepting that the pre-parser's three codes are then + covered only by their own characterization tests. §5.5 scopes Law A to + sources whose first line is a valid `swang 1` header — the header is the + premise, and only the body varies. One of the three was worse than merely + out of scope: `swang 2` is recorded as `SWG0001`, and SWG-4A-06 exists to + make `swang 2` supported, so a baseline built to protect that task would + have failed on the task's own correct behaviour and looked like a safety + net working. Every corpus source must now satisfy + `header_level(source) == Ok(1)`, because a domain that lives in a comment + is a domain that drifts. + +- 2026-08-29 — In the context of the level-1 boundary guard, facing an + exemption that looked harmless, we decided to **scan `syntax.rs` and + permit only its bare `mod limits;` line**, and against exempting the file + that declares the module, to achieve a guard that still holds when 4A-06 + arrives, accepting a slightly fussier stripper. `syntax.rs` is the crate's + public re-export point and the obvious home for level dispatch, so it is + simultaneously the file that must name the module and the file where a + budget consulted before the level 1/2 branch would silently become a + level-1 bound. Exempting it made the single most dangerous location the + one nobody watched. Level-2-specific modules join the exempt list one at a + time as 4A-06 creates them; shared dispatch never joins it. + +- 2026-08-29 — In the context of a budget with no caller yet, facing + `pub(crate)` limit fields and a `Clone` counter, we decided to **seal the + mechanism before the first caller exists**, to achieve a type shape that + enforces what the documentation already promised, accepting that tests + need a `#[cfg(test)]` constructor to reach scaled bounds. Public fields + would have let a future call site write `Level2Budget::new( + Level2ResourceLimits { tokens: u64::MAX, .. })` and satisfy every word of + §5.11 while meaning none of it; a `Clone` on a running counter lets the + same budget be spent twice, which is precisely what the type's own doc + comment said must not happen. The cheapest moment to close both doors is + while closing them breaks nothing. + +- 2026-08-30 — In the context of SWG-INF-06's external review, facing a Codex + finding that `MAX_TOKENS = 4_000_000` had an unstated representation + assumption, we decided to **keep the number and bind it to a level-2 + token-storage contract**, and against lowering it, to achieve a bound whose + memory derivation is written down, accepting that 4A-06 now owes a proof + its lexer would otherwise have been free to skip. The finding was correct + on every premise: `griff-swang` is a direct dependency of the Cockpit, + which CI builds for `wasm32-unknown-unknown`; level 1's `Token` owns a + `String` each — the lexer allocates one even for a single `{` — and + measures 40 bytes on a 64-bit host, so four million would exceed 150 MiB of + vector spine before millions of individual string allocations. §5.11 + promises a typed refusal rather than an allocation death, and on a browser + tab that promise would have failed. The recorded derivation was + "≈ 4 bytes per token at the byte cap", which ties `MAX_TOKENS` to + `MAX_SOURCE_BYTES` consistently but never asks what a retained token costs; + AGENTS.md requires the derivation be recorded, and a heap derivation was + not. Lowering the bound would have spent permanent acceptance-set budget to + accommodate a representation level 2 has not been written to inherit — + backwards, when the level-2 lexer does not exist and 4A-06 already owes the + first live wiring. So §5.11 states the contract as a budget rather than a + struct layout — no per-token owned lexeme storage, at most 12 bytes + retained per token on `wasm32`, text recovered from the span, or a strictly + stronger representation such as streaming — leaving a lexer free to do + better and forbidden only from doing worse. If 4A-06's measured `wasm32` + behaviour disproves the derivation, that is the moment to lower the bound, + still inside §5.11's deadline. + +- 2026-08-30 — In the context of the same review, facing a CodeRabbit + merge-risk note, we decided that **diagnostic exhaustion is terminal and + idempotent**, and against deferring the fix to 4A-06, to achieve a running + resource state that still describes reality after it is spent, accepting a + small behavioural change to a mechanism with no caller. Against a cap of + two, `diagnostics()` climbed to 7 across repeated admissions and the + reported `needed` count grew with it, describing an ever-larger + hypothetical parse for a parse already terminated. Every call returned + `Err`, so no caller obeying the contract could exceed the declared maximum + — which made the defect latent rather than absent, and latent is not the + same as correct. `admit_token` already pinned the matching law, that a + refused thing does not advance admitted state; the diagnostic axis differs + only in that its terminal refusal genuinely occupies the final slot, so the + rule is saturation rather than no-effect. Deferring it would have added one + more item to 4A-06's growing pile of inherited archaeology for no reason + other than that nobody could hit it yet. + +- 2026-08-30 — In the context of SWG-INF-06's **prior-art survey**, which + AGENTS.md requires before anything non-trivial and requires recorded here, + facing two independent designs carried by one task, we decided to adopt one + idea from each lineage and to refuse the central convenience of the first, + to achieve a baseline and a budget that borrow proven shapes instead of + invented ones, accepting that neither lineage settles the numbers this task + had to choose. The survey was owed and was missing; a task about evidence + should not be the one that skips its own admission gate. + + **The frozen baseline — `insta` and `expect-test`.** Both are the Rust + reference-test prior art, and both share the shape adopted here: an + expected artifact checked into the tree, recompared on every run, with a + diff as the failure. What is deliberately *not* adopted is the feature each + is built around. `insta` ships an accept/review workflow — `cargo insta + review`, `cargo insta accept`, `INSTA_UPDATE` — and `expect-test` exists so + that setting `UPDATE_EXPECT` rewrites the expectation in place. That is + correct for a snapshot of *current* behaviour, where re-recording is the + normal workflow and the artifact is a convenience. The Law A baseline is a + different object: the historical left-hand side of a comparison whose + right-hand side does not exist yet, stamped with the commit that produced + it. An updater would let the side under test rewrite the side it is tested + against, so this artifact is compare-only and regeneration is a deliberate + reviewed act in a detached worktree at the producer commit. The idea is + borrowed; the ergonomics are refused, and refused for a stated reason. + + **The resource bounds — `serde_json` and `rustc`.** The adopted idea is + admission *during* the descent rather than recovery after it: `serde_json` + carries a recursion limit checked as it parses, and its escape hatch is + documented with the warning that a caller who disables it must protect + against stack overflow by other means. That is precisely the claim §5.11 + makes — a crossed budget is a typed refusal, never an allocation death — + and it is why every axis here is admitted before the thing it counts + exists. `rustc`'s `recursion_limit` supplies the other half: a compiler may + *declare* a bound as part of its contract instead of discovering it at + runtime, which is what §5.11's before-the-first-accepted-program deadline + formalises. Neither supplies four axes or their values. `serde_json` bounds + one axis, `rustc` another; the byte, token, depth and diagnostic set, their + counting semantics, and the `wasm32` storage contract are this task's own, + derived in the entries above. Recording a survey is not a licence to claim + more inheritance than there is: nothing in `insta` taught this task to + count tokens. + +- 2026-08-30 — In the context of the level-1 boundary guard, facing a Codex + finding that the bare token `limits` fails the build on ordinary English, + we decided to **match qualified paths rather than a generic word**, to + achieve a guard specific enough to survive contact with future code, + accepting that the witness stays lexical and can still be fooled by + `limits::` inside a string. Reproduced before changing anything: a string + literal reading "no limits apply here" and a trailing `// … limits` + comment each failed CI, while a comment-only line was correctly ignored. + `limits` is an ordinary word and an ordinary identifier, and a guard that + blocks a build over prose is a guard the next person weakens — at which + point the frozen level loses its protection for a reason unrelated to the + frozen level. Every real route to the module carries `::`, so `limits::` + keeps the teeth: the original planted probe in `syntax.rs` is still caught. + Guarding against `limits::` in a string would need a Rust parser, which + costs more than this boundary is worth; the trade is stated in the test + rather than left for someone to discover. diff --git a/docs/swang/exact-score-text.md b/docs/swang/exact-score-text.md index eab80f2..146a832 100644 --- a/docs/swang/exact-score-text.md +++ b/docs/swang/exact-score-text.md @@ -1019,7 +1019,8 @@ same thing: | `SWG0403` | required word missing from a construct | | `SWG0404` | word repeated within a construct | -Four codes are new, because these failures do not exist at level 1: +Four codes are new to the grammar, because these failures do not exist at +level 1: | Code | Meaning | | --- | --- | @@ -1028,7 +1029,14 @@ Four codes are new, because these failures do not exist at level 1: | `SWG0507` | the tempo is a reduced fraction the canonical model cannot construct (H1's third branch) | | `SWG0508` | any string-literal escape fault — malformed (`\q`, an unterminated `\u{`) **and** valid-but-non-canonical (`\u{0a}` where `\n` is canonical, uppercase hex, leading zeros). Escapes are `SWG0508`'s alone; `SWG0505` never claims one | -No block is reserved beyond these four. Errors that do not exist yet get +One further code is level 2's, earned outside this document by the parser +resource gate rather than by the grammar: + +| Code | Meaning | +| --- | --- | +| `SWG0509` | a declared level-2 input budget was exceeded — source bytes, tokens, nesting depth, or diagnostics. One code for four axes, because they carry one meaning; the message names the axis, the declared limit, and what the parse would have needed. Declared by SWG-INF-06, defined in spec §5.11 | + +No block is reserved beyond these five. Errors that do not exist yet get numbers when they do. ## 7. Fixture matrix diff --git a/docs/swang/foundation-backlog.md b/docs/swang/foundation-backlog.md index 7e87a09..15335c7 100644 --- a/docs/swang/foundation-backlog.md +++ b/docs/swang/foundation-backlog.md @@ -109,7 +109,7 @@ recorded in `decisions.log.md` if reversed. | SWG-INF-03 | Split `syntax.rs` without behaviour change *(done)* | code | INF-01 | | SWG-INF-04 | Replace `ProgramSpans` with a `SourceMap` *(done)* | code | INF-03 | | SWG-INF-05 | Deterministic multi-error recovery | code | INF-04 | -| SWG-INF-06 | Parser resource gate and differential harness | code | INF-02, INF-03 | +| SWG-INF-06 | Parser resource gate and Law A baseline *(done)* | code | INF-02, INF-03 | | SWG-4A-01 | Normative exact-score-text grammar *(done)* | docs | INF-02 | | SWG-CORE-01 | Fixed-width migration for the three `usize` fields *(done)* | code | 4A-01 | | SWG-CORE-02 | Decide whether the canonical newtypes seal their fields | docs | 4A-01 | @@ -207,9 +207,11 @@ the spec. `spec.md` §1 and §3 are byte-unchanged, verified by digest. What §5 settles, for downstream tasks that need the answer: - **Law A adopted, strengthened.** A build supporting `1..=N` treats every - `swang 1` source — *including invalid ones* — exactly as a level-1-only - build did, compared on all seven observables: verdict, AST, canonical - bytes, diagnostic code, message, span, and order. A level-2 keyword in a + source whose first line is a **valid `swang 1` header** — *including those + with invalid bodies* — exactly as a level-1-only build did, compared on all + seven observables: verdict, AST, canonical bytes, diagnostic code, message, + span, and order. The valid header is the premise, not part of what varies: + §5.5 does not reach a source the frozen pre-parser refuses outright. A level-2 keyword in a `swang 1` script raises the `SWG0401` level 1 already raised, never a friendlier "requires language level 2". - **Law B rejected.** A `swang 2` header over a level-1 body is not valid by @@ -388,39 +390,168 @@ Acceptance: unchanged** — recovery adds diagnostics, it never renames the first one; - accept/reject verdicts are unchanged for every existing fixture; - adversarial input does not go quadratic (assert a bounded step count, not - a wall-clock time). - -### SWG-INF-06 — Parser resource gate and differential harness + a wall-clock time); +- **inherited from SWG-INF-06** — recovery is the first thing that can + approach the declared diagnostic bound, so it consults + `Level2Budget::admit_diagnostic` before appending. This task's own cap + (32 above) may be stricter than §5.11's 256; it may never exceed it, and + the terminal `SWG0509` counts toward whichever cap applies; +- **inherited from SWG-INF-06** — this entry's "the first diagnostic of + every existing single-error golden is unchanged" bullet is **not** + sufficient once Law A is stated. §5.10 freezes level 1's diagnostic + *order*, and the Law A baseline records the whole sequence, not its first + element; a level-1 parse that begins returning three diagnostics where it + returned one has changed a frozen level's released output even though the + first one is untouched. INF-05 must therefore say plainly whether recovery + is level-2-only, or how a multi-diagnostic level-1 parse stays inside + §5.10 — and reconcile the acceptance bullet above with the seven-observable + contract before writing code against it. + +### SWG-INF-06 — Parser resource gate and Law A baseline *(done)* **Kind:** code. **Depends on:** INF-02, INF-03 -Add bounded-input laws: maximum source bytes, maximum token count, maximum -nesting depth, maximum diagnostics, no recursion overflow, and a canonical -formatter that only ever writes from a checked AST. - -**Constraint from the freeze, settled by INF-02:** rejecting an input that -level 1 previously accepted is an observable semantic change to a frozen -level, so spec §5.11 puts every declared bound at **level 2 only**, to be -declared before level 2's first accepted program. Level 1 keeps its -acceptance set exactly; a level-1 run may still die of exhaustion, but that -is a runtime outcome and never becomes a typed refusal. This task therefore -adds no level-1 limit at all — the earlier "or prove the bound exceeds every -representable level-1 program" branch is closed. - -Differential harness: the pre-refactor parser output is compared with the -refactored parser over the fixture set and the fuzz corpus on AST, canonical -text, diagnostic code, and owning span. +Landed as `swang/src/syntax/limits.rs` (the budget), `law_a_baseline.rs` +plus its frozen `.golden` (the differential), `level_two_budget_boundary.rs` +(the level-1 guard), spec §5.11's four declared bounds, and `SWG0509`. -Fuzz oracles (extending `swang_parse`): +**The bounds are level 2's alone, and they are declared, not enforced here.** +Spec §5.11 puts every declared bound at level 2 only: rejecting an input that +level 1 previously accepted is an observable change to a frozen level. A +level-1 run may still die of exhaustion, but that is a runtime outcome and +never becomes a typed refusal. This task therefore adds no level-1 limit at +all — the earlier "or prove the bound exceeds every representable level-1 +program" branch is closed, and two witnesses hold the line: one reads the six +level-1 modules, the other walks every shipped `.rs` under `swang/src`. -```text -parse never panics -Err carries at least one diagnostic -every code matches SWG\d{4} -every span lies within the source -format(parse(format(ast))) is a fixed point -a limit breach is a typed error, not an allocation death -``` +| Axis | Limit | +| --- | --- | +| source bytes | `16_777_216` (16 MiB) | +| tokens | `4_000_000` | +| nesting depth | `64` | +| diagnostics | `256` | + +Counting semantics are in spec §5.11 and are part of the declaration. Depth +and diagnostics are **forward reservations**: the exact-score grammar has no +recursive production and today's parser returns one diagnostic, so neither +can currently be approached. They are declared anyway because §5.11's +deadline — before level 2's first accepted program — is an admission rule +**stricter than the freeze boundary**, not a consequence of it: by §5.3 +level 2 stays provisional until Phase 4A is accepted, so a later bound would +still predate the freeze, and §5.11 forbids it regardless because programs +are by then already written against the level. + +Level 2 is unreachable on this build, so the budget has **no live caller**. +Wiring a gate into a parser that does not exist would be the fake half of the +work: the mechanism ships here, the wiring is SWG-4A-06's, and that +obligation is written into its acceptance rather than left to memory. + +**The differential harness, corrected.** This entry used to ask for the +pre-refactor parser to be diffed against the refactored one. INF-03 landed +and deleted that parser, and its own acceptance already discharged the +comparison — byte-identical §3.1 reference, identical test counts, no edited +expected value, and its own 168-mutation round. There is no historical +implementation left in the tree to compare against, and resurrecting one +purely to diff it would be theatre. + +The live differential is Law A (spec §5.5): a build supporting `1..=N` must +treat every source whose first line is a valid `swang 1` header, **invalid +bodies included**, exactly as a +level-1-only build did, on verdict, AST, canonical bytes, diagnostic code, +message, span, and order. Today `N` is 1, so that comparison has nothing on +its right-hand side; when 4A-06 supplies one, the level-1-only build will be +gone exactly as the pre-refactor parser is gone now. So the left-hand side is +recorded now, as a frozen artifact naming the commit that produced it +(`c44313c`, generated in a detached worktree at that commit), and it is +**compare-only** — there is deliberately no snapshot-update path. + +The AST observation is a test-owned exhaustive projection: not `Debug`, not +serde, and not the formatter's output, because canonical bytes and the AST +have to be two witnesses rather than one wearing two hats. Every struct is +destructured with no `..` and every enum matched with no wildcard. + +**The domain is a witness.** §5.5 scopes Law A to sources whose first line is +already a valid `swang 1` header; only the body varies. The corpus first +included three sources the pre-parser refuses outright — `swang 2`, a +malformed header, and a BOM — and the first of those was actively harmful: +its recorded refusal is `SWG0001`, and 4A-06 exists to make `swang 2` +supported, so the artifact built to protect 4A-06 would have declared +4A-06's whole purpose a regression. Every corpus source now has to satisfy +`header_level(source) == Ok(1)`, so the mistake cannot recur. The +pre-parser's own `SWG0001`–`SWG0003` are unreachable inside the domain and +keep their characterization tests beside the frozen pre-parser. + +Extent, stated rather than implied: 47 cases reaching 35 distinct +`(code, message)` refusals, both verdicts, every level-1 enum variant, both +states of every optional, and all eleven codes reachable inside the domain. +The checked-in `swang_parse` seed is included as an input subset. Coverage +by *code* proved weaker than coverage by *production site* — `SWG0403` is +raised from four places — and a falsification probe survived until the +corpus grew to reach them; `the_corpus_pins_the_extent_of_its_own_sample` +now records the number so the sample cannot shrink quietly. + +**Fuzz oracles.** `swang_parse` asserted `starts_with("SWG")`, which accepted +`SWG`, `SWGxyz`, and `SWG12345` as registry codes; it now asserts the one +shape the registry has, with no regex dependency. The other listed oracles +were already present. The limit-breach oracle is **not** claimed here: public +parsing cannot reach level 2 on this build, so an end-to-end breach oracle +would cover an execution path the binary cannot enter. It belongs to 4A-06. + +**The formatter clause is not this task's.** The original entry also asked +for "a canonical formatter that only ever writes from a checked AST". The +exact writer lane is complete, and checked lowering from text to a valid +`Score` is SWG-4A-09's stated contract. A resource-gate change has no +business becoming another formatter-validation layer, so that clause is +recorded as already owned rather than implemented here. + +**The mechanism is sealed.** `Level2ResourceLimits` has private fields and +`declared()` as its only production constructor, so no future caller can +satisfy the contract with bounds of its own choosing, and `Level2Budget` is +neither `Copy` nor `Clone`, so a running counter cannot be duplicated and +spent twice. The level-1 guard scans `syntax.rs` too, permitting only the +bare `mod limits;` line: that file is the crate's re-export point and +4A-06's natural home for shared dispatch, and a budget consulted before the +level branch is a level-1 bound whatever file it lives in. + +**The token bound is paired with a storage contract, not lowered.** Codex +found that `MAX_TOKENS` had an unstated representation assumption: level 1's +`Token` owns a `String` each, so four million of them would exceed 150 MiB +of vector spine before millions of string allocations — and `griff-swang` is +a direct dependency of the Cockpit, which is built for +`wasm32-unknown-unknown`. Rather than shrink a permanent language bound to +fit a representation level 2 need not inherit, spec §5.11 now carries the +derivation and the normative storage contract: no per-token owned lexeme +storage, at most 12 bytes retained per token on `wasm32`, text recovered +from the span. 4A-06 inherits the obligation to prove it. + +**Diagnostic exhaustion is terminal.** CodeRabbit found that the budget kept +counting after it was spent — `diagnostics()` reached 7 against a cap of 2. +A caller obeying `Err` never exceeded the cap, so the defect was latent, not +absent; a type that calls itself a running resource state has to report one. +The last slot is now consumed once and later admissions are refused +identically, giving the diagnostic axis the law `admit_token` already had. + +**Two findings from the final external round.** Codex observed that the task +had no recorded prior-art survey, which AGENTS.md requires for anything +non-trivial — a real gap, and one this PR would otherwise have carried while +marking INF-06 done. The survey is now in the decision log: `insta` and +`expect-test` for the checked-in reference artifact (shape adopted, updater +deliberately refused), `serde_json` and `rustc` for admission during the +descent and for declaring a bound as contract. Codex also found the level-1 +guard failing on the ordinary word `limits` in a string or a trailing +comment; it now matches `limits::`, with witnesses for five benign forms and +six real ones. + +Falsification: 16 probes, 0 survivors — an off-by-one at each of the four +caps, a token counted despite being refused, a no-op depth counter, a +diagnostic cap returning one item too many, two malformed registry codes, a +reworded frozen message, a formatter spacing change, an axis reporting +another axis's word, a depth breach pointing somewhere fixed, a diagnostic +breach understating what it needed, a production back door to arbitrary +limits, a budget mention in `syntax.rs`, and accounting that runs on past a +terminal diagnostic breach. Seven of them are recorded as SURVIVED before a +review commit and CAUGHT after, rather than as though the first suite had +caught them. --- @@ -679,7 +810,28 @@ Acceptance: - a level-2 `score` reaches the exact parser; - an unknown newer level is still refused by the frozen pre-parser; - dispatch routes to one level's entry point and never branches inside a - shared grammar (spec §5.4). + shared grammar (spec §5.4); +- **inherited from SWG-INF-06** — before the first successful `swang 2` + result, level-2 dispatch constructs and consults the INF-06 budget: + source bytes checked pre-lex, token accounting during lexing, and + structural-depth accounting during parsing. No level-1 path imports or + consults that budget, and `level_two_budget_boundary.rs` still passes; +- **inherited from SWG-INF-06** — `swang_parse` gains the end-to-end oracle + INF-06 could not honestly claim: a limit breach is a typed `SWG0509`, not + an allocation death. It lands here because this is the task that first + lets a fuzzed input reach a level-2 parser at all; +- **inherited from SWG-INF-06** — before the first successful `swang 2`, the + level-2 lexer proves the `MAX_TOKENS` heap derivation on the `wasm32` + frontend: retained tokens own no lexeme `String` or other per-token heap + allocation; a materialized token is at most 12 bytes on `wasm32`, or the + lexer proves a strictly smaller or streaming retained representation. + Token text is recovered from the source span. **Level 1's `String`-owning + `Token` is not the level-2 storage representation.** A witness is + required, not prose: for a materialized representation a compile-time size + assertion plus tests showing text is source-sliced suffices — there is no + need to allocate four million tokens to admire the fan spinning. + Preregistered falsification probe: *adding owned lexeme text to the + level-2 token* **must be CAUGHT**. ### SWG-4A-07 — Parser: exact scalar types @@ -1342,10 +1494,13 @@ INF-01 status sync (done) │ surface over it, not another slice of it │ └─→ 4A-02 → INF-04 → INF-06 → 4A-06 parser skeleton - (done) (done) ↑ - next — level 2's input bounds must - be declared before its first - accepted program (§5.11) + (done) (done) (done) ↑ + next — it inherits + INF-06's live wiring and + the end-to-end breach + oracle, and Law A's + frozen baseline is what + its dispatch must keep -> 4A-02..4A-09 writer / parser / builder -> 4A-10..4A-14 dump / verify / laws / fuzz -> 4B corpus acceptance diff --git a/docs/swang/spec.md b/docs/swang/spec.md index b9c612b..89cd97f 100644 --- a/docs/swang/spec.md +++ b/docs/swang/spec.md @@ -812,3 +812,121 @@ Level 2 may declare bounds — source bytes, token count, nesting depth, diagnostic count — as part of its own contract, and must declare them **before its first accepted program**, not after the first crash. That is the space the parser resource gate works in. + +#### The four declared level-2 bounds + +SWG-INF-06 declares them, ahead of level 2's first accepted program: + +| Axis | Limit | What is counted | +| --- | --- | --- | +| source bytes | `16_777_216` (16 MiB) | UTF-8 bytes of the complete source, header included | +| tokens | `4_000_000` | tokens the level-2 lexer emits after the frozen header pre-parser; end of input is not a token | +| nesting depth | `64` | simultaneously open structural `{ … }` constructs; the `score` root is depth 1, and a `[ … ]` scalar list adds no structural depth | +| diagnostics | `256` | the most one level-2 parse attempt may return, the terminal budget diagnostic included | + +The counting semantics are part of the declaration. Without them `64` and +`4_000_000` are decorative numerology: a token budget that counts differently +is a different budget. + +Each bound is checked **before** the thing it counts exists — the source +before lexing, each token before it is stored, each block on entry, each +diagnostic before it is appended. Checking `tokens.len()` after lexing four +million tokens is not a resource gate; it is an obituary written after the +allocation. + +#### The token bound's derivation, and the level-2 token-storage contract + +`MAX_TOKENS` is derived against the **level-2 token-storage contract below**, +not against level 1's lexer representation. The two must not be confused: a +token budget is a memory bound only in combination with a statement of what a +retained token costs. + +A retained level-2 token carries only its classification and its source +location, or uses a representation with an equal-or-smaller retained +footprint. Lexeme text is recovered from the immutable source through its +`Span` — it is never copied into the token. + +```text +on wasm32: + Span = 8 bytes + compact kind + Span token <= 12 bytes + 4,000,000 retained tokens <= 48,000,000 bytes ~ 45.8 MiB nominal + + 16 MiB source + full token spine ~ 61.8 MiB + before parser and allocator overhead +``` + +The normative requirement is a **budget, not a struct layout**: + +> No per-token owned lexeme storage. A retained level-2 token occupies at +> most 12 bytes on `wasm32`, or the lexer uses a strictly stronger +> representation — streaming, for instance — that retains less. + +Stated that way, a level-2 lexer is free to do better than a +`{ kind, span }` pair, and is forbidden only from doing worse. + +This matters because `griff-swang` is a direct dependency of the Cockpit, +which is built and exercised for `wasm32-unknown-unknown`. Level 1's `Token` +owns a `String` per token — the lexer allocates one even for a single `{` — +which at 40 bytes of spine per token on a 64-bit host would put four million +tokens past 150 MiB of vector alone, before millions of individual string +allocations. On a memory-constrained browser tab a budget of that shape +would be reached by exhaustion rather than by refusal, which is precisely +what §5.11 exists to prevent. The bound is therefore paired with a storage +contract rather than lowered to accommodate a representation level 2 does +not have to inherit. + +If a level-2 lexer's measured `wasm32` behaviour ever disproves this +derivation, the time to lower the bound is then — still before level 2's +first accepted program, and still inside this section's deadline. + +#### Two of the four are forward reservations + +The exact-score grammar §5.7 allocates to level 2 has no recursive +production — `score`, `track`, `voice`, `group`, `note`, `position`, +`evidence` bottoms out — so **it cannot currently approach depth 64**, and +today's parser maps each error into a one-element vector, so it cannot +approach 256 diagnostics. Both are declared anyway, and both are +reservations: they are not evidence that today's grammar has a +recursive-stack hazard or a diagnostic flood. + +They are declared now because this section leaves no later opportunity. The +deadline above — before level 2's first accepted program — is an **admission +rule stricter than the freeze boundary**, not a consequence of it. By §5.3 +level 2 remains provisional until Phase 4A is accepted, so a bound +introduced after the first accepted program would still predate the freeze; +§5.11 forbids it regardless, because by then programs are being written +against the level and a new bound would start rejecting them. Declaring the +bounds costs nothing today; omitting them spends the option permanently. A +future recovery implementation may stop well short of 256 — it simply may +not exceed it. + +#### The breach diagnostic + +A crossed budget is a **typed refusal**, never an allocation death. It is +`SWG0509` — one code for all four axes, because they carry one meaning, and +§5.10 forbids one number meaning two things. The message names the axis, the +declared limit, and what the parse would have needed. + +Precedence, so a breach cannot be mistaken for a grammar error: + +```text +malformed or unsupported header -> SWG000x, before any budget is consulted +supported `swang 2`, source too long -> SWG0509, before lexing +token budget crossed -> SWG0509, before structural parsing +depth budget crossed on entering a legal construct -> SWG0509 +diagnostic budget exhausted -> terminal SWG0509, within the 256 total +``` + +Where a breach points: + +| Breach | Location | +| --- | --- | +| source bytes | the level/header span — no body token has been admitted | +| tokens | the token that crosses the budget | +| nesting depth | the opening token of the construct that would exceed it | +| diagnostics | the diagnostic whose production exhausts the budget | + +A source containing nine hundred `{` characters does not have to become a +depth-budget error if the grammar rejects it structurally first. The +resource checker must not understand more grammar than the parser does. diff --git a/fuzz/fuzz_targets/swang_parse.rs b/fuzz/fuzz_targets/swang_parse.rs index e76df95..b0306e3 100644 --- a/fuzz/fuzz_targets/swang_parse.rs +++ b/fuzz/fuzz_targets/swang_parse.rs @@ -11,8 +11,8 @@ //! * `header_level`: `Ok(level)` in `1..=LANGUAGE_LEVEL` xor a typed //! diagnostic. //! * `parse`: `Ok(Program)` xor a non-empty `Vec`. -//! * Every diagnostic carries an `SWG`-prefixed registry code and a span -//! inside the source (`start <= end <= len`). +//! * Every diagnostic carries a registry code of exactly the shape +//! `SWG\d{4}` and a span inside the source (`start <= end <= len`). //! * On `Ok`: `format` emits canonical text that reparses to the same AST //! (law 3) and is its own fixed point (law 2). @@ -20,11 +20,18 @@ use griff_swang::syntax::{format, header_level, parse, Diagnostic, LANGUAGE_LEVE use libfuzzer_sys::fuzz_target; /// The one diagnostic contract, applied to the header pre-parser and the -/// parser alike: a stable `SWG` registry code and a span inside the source. +/// parser alike: a registry code of exactly the shape `SWG\d{4}` and a span +/// inside the source. +/// +/// `starts_with("SWG")` was the weaker form this oracle shipped with, and it +/// accepted `SWG`, `SWGxyz`, and `SWG12345` as registry codes. The registry +/// has one shape (spec §1.5); the oracle now asserts that shape rather than +/// its first three bytes. No regex dependency is needed to say so. fn assert_diagnostic(d: &Diagnostic, len: u32) { + let digits = d.code.strip_prefix("SWG"); assert!( - d.code.starts_with("SWG"), - "stable registry code, never ad hoc: {}", + digits.is_some_and(|rest| rest.len() == 4 && rest.bytes().all(|b| b.is_ascii_digit())), + "a registry code is exactly `SWG` and four digits, never ad hoc: {}", d.code ); assert!( diff --git a/swang/src/syntax.rs b/swang/src/syntax.rs index ab6280b..d03c274 100644 --- a/swang/src/syntax.rs +++ b/swang/src/syntax.rs @@ -58,6 +58,7 @@ mod diagnostic; mod format; mod header; mod lexer; +mod limits; mod parser; mod source_map; mod span; diff --git a/swang/src/syntax/limits.rs b/swang/src/syntax/limits.rs new file mode 100644 index 0000000..678e412 --- /dev/null +++ b/swang/src/syntax/limits.rs @@ -0,0 +1,325 @@ +//! The level-2 input budget (SWG-INF-06, spec §5.11). +//! +//! # Level 2 only, and the type is named for it +//! +//! Level 1's acceptance set is frozen (§5.5). A parser-wide bound that +//! rejected a source level 1 accepts would be an observable change to a +//! frozen level, so §5.11 puts every declared bound at level 2 alone. A +//! level-1 run may still die of exhaustion — that is a runtime outcome, and +//! it never becomes a typed refusal. Nothing in this module may be reached +//! from a level-1 path, and the type carries `Level2` in its name so that a +//! call site which forgot is visible at a glance rather than after a +//! bisect. +//! +//! # A live counter, not an audit +//! +//! Every axis is admitted *before* the thing it counts is built. Checking +//! `tokens.len()` after lexing four million tokens is not a resource gate; +//! it is an obituary written after the allocation. The enforcement order the +//! level-2 parser owes this module: +//! +//! ```text +//! frozen header pre-parser +//! -> resolve the supported level +//! -> level 1: never consult this module +//! -> level 2: admit_source before lexing +//! admit_token while lexing, before storing the token +//! enter_block on entering a structural `{ ... }` +//! admit_diagnostic before appending another diagnostic +//! ``` +//! +//! # What the counters count +//! +//! - **source bytes** — UTF-8 bytes of the complete source, header included; +//! - **tokens** — tokens emitted by the level-2 lexer after the frozen +//! header pre-parser. End of input is not a token; +//! - **nesting depth** — simultaneously open structural `{ ... }` +//! constructs. The `score` root is depth 1. A `[ ... ]` scalar list is one +//! value carried by one word, so it adds no structural depth; +//! - **diagnostics** — the most one level-2 parse attempt may return. A +//! terminal budget diagnostic counts toward that total, so the last slot +//! is reserved for it rather than spent and then exceeded. +//! +//! # Two of the four are forward reservations +//! +//! The exact-score grammar Phase 4A admits has no recursive production — +//! `score`, `track`, `voice`, `group`, `note`, `position`, `evidence` +//! bottoms out — so it cannot approach depth 64, and today's parser maps +//! each error into a one-element vector, so it cannot approach 256 +//! diagnostics. Both limits are declared anyway, because §5.11 requires +//! declaration *before* level 2's first accepted program. That deadline is +//! §5.11's own admission rule, and it is **stricter than the freeze**: by +//! §5.3 level 2 stays provisional until Phase 4A is accepted, so a bound +//! added after the first accepted program would still predate the freeze — +//! §5.11 forbids it anyway, because programs written against a provisional +//! level are already running. They are reservations, not evidence of a +//! stack-overflow hazard in today's grammar. +//! +//! # No caller yet, on purpose +//! +//! Level 2 is unreachable on this build — `header_level` refuses `swang 2` +//! — so this module has no live caller. SWG-4A-06 owes it one: level-2 +//! dispatch must construct and consult this budget before its first +//! successful level-2 parse. Wiring a gate into a parser that does not exist +//! would be the fake half of the work, so the mechanism ships here and the +//! wiring ships there. +#![allow(dead_code)] + +use super::diagnostic::Diagnostic; +use super::span::Span; + +/// UTF-8 bytes of source a level-2 parse may read — exactly `16 MiB`. +pub(crate) const MAX_SOURCE_BYTES: u64 = 16_777_216; + +/// Tokens a level-2 lex may emit. +pub(crate) const MAX_TOKENS: u64 = 4_000_000; + +/// Simultaneously open structural blocks a level-2 parse may hold. +pub(crate) const MAX_NESTING_DEPTH: u32 = 64; + +/// Diagnostics one level-2 parse attempt may return, terminal budget +/// diagnostic included. +pub(crate) const MAX_DIAGNOSTICS: u32 = 256; + +/// The four declared level-2 bounds. +/// +/// The fields are private and there is no production constructor but +/// [`Level2ResourceLimits::declared`]. A declared bound that a caller may +/// replace is not a declared bound: `Level2Budget::new(Level2ResourceLimits +/// { tokens: u64::MAX, .. })` would satisfy every word of the contract while +/// meaning none of it. Tests reach the scaled constructor below, which is +/// `#[cfg(test)]` and therefore cannot appear in a shipped call site. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct Level2ResourceLimits { + /// UTF-8 bytes of the complete source, header included. + source_bytes: u64, + /// Tokens after the frozen header pre-parser; end of input is not one. + tokens: u64, + /// Simultaneously open structural `{ ... }` constructs. + nesting_depth: u32, + /// Diagnostics one parse attempt may return. + diagnostics: u32, +} + +impl Level2ResourceLimits { + /// The bounds spec §5.11 declares. The only way to build these outside + /// a test. + pub(crate) const fn declared() -> Self { + Self { + source_bytes: MAX_SOURCE_BYTES, + tokens: MAX_TOKENS, + nesting_depth: MAX_NESTING_DEPTH, + diagnostics: MAX_DIAGNOSTICS, + } + } + + /// Scaled-down bounds, so a test can prove an exact off-by-one without + /// allocating the declared caps. Test-only on purpose: see the type's + /// documentation. + #[cfg(test)] + pub(crate) const fn scaled( + source_bytes: u64, + tokens: u64, + nesting_depth: u32, + diagnostics: u32, + ) -> Self { + Self { + source_bytes, + tokens, + nesting_depth, + diagnostics, + } + } +} + +/// Which budget a refusal is about. One code, four axes — they share a +/// meaning, so they share `SWG0509`, and the message names the axis. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Axis { + SourceBytes, + Tokens, + NestingDepth, + Diagnostics, +} + +impl Axis { + const fn word(self) -> &'static str { + match self { + Self::SourceBytes => "source bytes", + Self::Tokens => "tokens", + Self::NestingDepth => "nesting depth", + Self::Diagnostics => "diagnostics", + } + } +} + +/// Builds the one budget refusal, naming the axis, the declared limit, and +/// what the parse would have needed. +fn breach(axis: Axis, limit: u64, needed: u64, at: Span) -> Diagnostic { + Diagnostic { + code: "SWG0509", + span: at, + message: format!( + "level-2 {} budget exceeded: the declared limit is {limit}, this parse needed {needed}", + axis.word() + ), + } +} + +/// A level-2 parse's running resource state. +/// +/// Deliberately neither `Copy` nor `Clone`: it is a counter, and a +/// duplicated counter lets a caller spend the same budget twice by +/// advancing the copy. The documentation already promised that; the derives +/// used to contradict it. +#[allow( + missing_copy_implementations, + reason = "a running counter must not be silently duplicated" +)] +#[derive(Debug, PartialEq, Eq)] +pub(crate) struct Level2Budget { + limits: Level2ResourceLimits, + tokens: u64, + depth: u32, + diagnostics: u32, +} + +impl Level2Budget { + /// The budget every level-2 parse runs under: the bounds §5.11 declares, + /// with every counter at zero. This is the only constructor a shipped + /// call site can reach. + pub(crate) const fn declared() -> Self { + Self::over(Level2ResourceLimits::declared()) + } + + /// A fresh budget over the given bounds. Private, so the declared bounds + /// cannot be swapped out at a call site; `declared()` is the production + /// door and `#[cfg(test)]` code reaches this through [`Self::scaled`]. + const fn over(limits: Level2ResourceLimits) -> Self { + Self { + limits, + tokens: 0, + depth: 0, + diagnostics: 0, + } + } + + /// A budget over scaled-down bounds, for boundary tests. + #[cfg(test)] + pub(crate) const fn scaled(limits: Level2ResourceLimits) -> Self { + Self::over(limits) + } + + /// Tokens admitted so far. + pub(crate) const fn tokens(&self) -> u64 { + self.tokens + } + + /// Structural blocks currently open. + pub(crate) const fn depth(&self) -> u32 { + self.depth + } + + /// Diagnostics accounted for, terminal one included. + pub(crate) const fn diagnostics(&self) -> u32 { + self.diagnostics + } + + /// Admits the whole source, **before lexing**. + /// + /// `at` is the caller's location for a refusal: no body token has been + /// admitted yet, so the level/header span is the only honest place to + /// point. + /// + /// # Errors + /// `SWG0509` when the source is longer than the declared byte budget. + pub(crate) fn admit_source(&self, source: &str, at: Span) -> Result<(), Diagnostic> { + let bytes = u64::try_from(source.len()).unwrap_or(u64::MAX); + if bytes > self.limits.source_bytes { + return Err(breach( + Axis::SourceBytes, + self.limits.source_bytes, + bytes, + at, + )); + } + Ok(()) + } + + /// Admits one token, **before the lexer stores it**. A refused token is + /// not counted: the budget records what it granted, never what it + /// turned away. + /// + /// # Errors + /// `SWG0509` at the token that crosses the budget. + pub(crate) fn admit_token(&mut self, at: Span) -> Result<(), Diagnostic> { + let needed = self.tokens.saturating_add(1); + if needed > self.limits.tokens { + return Err(breach(Axis::Tokens, self.limits.tokens, needed, at)); + } + self.tokens = needed; + Ok(()) + } + + /// Opens one structural block. The `score` root is depth 1; a scalar + /// list never calls this. + /// + /// # Errors + /// `SWG0509` at the opening token of the block that would exceed the + /// depth. The refused block is not entered. + pub(crate) fn enter_block(&mut self, at: Span) -> Result<(), Diagnostic> { + let needed = self.depth.saturating_add(1); + if needed > self.limits.nesting_depth { + return Err(breach( + Axis::NestingDepth, + u64::from(self.limits.nesting_depth), + u64::from(needed), + at, + )); + } + self.depth = needed; + Ok(()) + } + + /// Closes the innermost structural block. Saturating, so an unbalanced + /// close cannot wrap the counter into a budget it never earned. + pub(crate) const fn leave_block(&mut self) { + self.depth = self.depth.saturating_sub(1); + } + + /// Accounts for one diagnostic, **before it is appended**. + /// + /// The cap is on what one parse attempt returns, and the terminal budget + /// diagnostic counts toward it, so the last slot is reserved for that + /// refusal instead of being spent on an ordinary diagnostic and then + /// exceeded. + /// + /// Exhaustion is **terminal and idempotent**: the last slot is consumed + /// once, and every later admission is refused identically without + /// advancing admitted state. `admit_token` states the same law — a + /// refused thing does not move the counter — and the only difference + /// here is that the terminal refusal itself genuinely occupies a slot. + /// A budget that kept counting after termination would report a parse + /// that never happened. + /// + /// # Errors + /// `SWG0509` — itself the last diagnostic the attempt may return. + pub(crate) fn admit_diagnostic(&mut self, at: Span) -> Result<(), Diagnostic> { + let needed = self.diagnostics.saturating_add(1); + if needed >= self.limits.diagnostics { + // Saturate rather than increment: `needed` is what a parse would + // have required to keep this diagnostic *and* still carry the + // terminal refusal, which is one past the cap however many times + // an ignored `Err` is retried. + self.diagnostics = self.limits.diagnostics; + return Err(breach( + Axis::Diagnostics, + u64::from(self.limits.diagnostics), + u64::from(self.limits.diagnostics.saturating_add(1)), + at, + )); + } + self.diagnostics = needed; + Ok(()) + } +} diff --git a/swang/src/syntax/tests.rs b/swang/src/syntax/tests.rs index 9baf954..a58fd28 100644 --- a/swang/src/syntax/tests.rs +++ b/swang/src/syntax/tests.rs @@ -632,3 +632,268 @@ fn parse_format_roundtrips_any_constructible_program() { let roundtripped = parse(&format(&program)).expect("formatted text parses"); assert_eq!(roundtripped, program); } + +/// SWG-INF-06: the level-2 resource budget's contract, stated before it +/// exists. +/// +/// Spec §5.11 puts every declared input bound at **level 2 only**: level 1's +/// acceptance set is frozen, so a parser-wide bound that rejected a source +/// level 1 accepts would be an observable change to a frozen level. Nothing +/// here may ever be consulted on a level-1 path, and the type is named for +/// that. +/// +/// The budget is a live counter, not a post-hoc audit. Checking +/// `tokens.len()` after lexing four million tokens is not a resource gate; +/// it is an obituary written after the allocation. So every axis is admitted +/// *before* the thing it counts is built. +mod level_two_budget { + use crate::syntax::limits::{ + Level2Budget, Level2ResourceLimits, MAX_DIAGNOSTICS, MAX_NESTING_DEPTH, MAX_SOURCE_BYTES, + MAX_TOKENS, + }; + use crate::syntax::{Diagnostic, Span}; + + /// Any location; these tests are about counting, not about pointing. + const AT: Span = Span { start: 0, end: 1 }; + + /// A scaled-down budget, so every boundary is testable at its exact + /// off-by-one without allocating the declared caps. + const fn small() -> Level2ResourceLimits { + Level2ResourceLimits::scaled(8, 3, 2, 2) + } + + /// A budget over the scaled bounds. + fn budget(limits: Level2ResourceLimits) -> Level2Budget { + Level2Budget::scaled(limits) + } + + #[test] + fn the_declared_limits_are_the_four_numbers_the_spec_names() { + assert_eq!(MAX_SOURCE_BYTES, 16_777_216, "exactly 16 MiB"); + assert_eq!(MAX_TOKENS, 4_000_000); + assert_eq!(MAX_NESTING_DEPTH, 64); + assert_eq!(MAX_DIAGNOSTICS, 256); + assert_eq!( + Level2ResourceLimits::declared(), + Level2ResourceLimits::scaled( + MAX_SOURCE_BYTES, + MAX_TOKENS, + MAX_NESTING_DEPTH, + MAX_DIAGNOSTICS + ), + "the declared bounds are exactly the four constants" + ); + } + + #[test] + fn a_source_of_exactly_the_limit_is_admitted() { + let budget = budget(small()); + budget + .admit_source("12345678", AT) + .expect("eight bytes is exactly the limit"); + } + + #[test] + fn one_byte_over_the_source_limit_is_refused() { + let budget = budget(small()); + let refusal = budget + .admit_source("123456789", AT) + .expect_err("nine bytes exceeds a limit of eight"); + assert_eq!(refusal.code, "SWG0509"); + } + + #[test] + fn the_source_limit_counts_utf8_bytes_not_characters() { + // Three two-byte characters are six bytes, not three. + let budget = budget(Level2ResourceLimits::scaled(5, 3, 2, 2)); + budget + .admit_source("ééé", AT) + .expect_err("six bytes exceeds a limit of five"); + } + + #[test] + fn the_declared_source_limit_is_the_one_actually_consulted() { + // The scaled budget proves the arithmetic; this proves the real + // number is wired to it rather than merely declared beside it. + let budget = Level2Budget::declared(); + let at_limit = "a".repeat(usize::try_from(MAX_SOURCE_BYTES).expect("16 MiB fits usize")); + budget + .admit_source(&at_limit, AT) + .expect("exactly the declared limit"); + let over = format!("{at_limit}a"); + budget + .admit_source(&over, AT) + .expect_err("one byte over the declared limit"); + } + + #[test] + fn the_token_budget_admits_exactly_its_limit_then_refuses() { + let mut budget = budget(small()); + for _ in 0..3 { + budget.admit_token(AT).expect("within the token budget"); + } + assert_eq!(budget.tokens(), 3); + let refusal = budget.admit_token(AT).expect_err("the fourth token"); + assert_eq!(refusal.code, "SWG0509"); + } + + #[test] + fn a_refused_token_is_not_counted() { + // The lexer asks before it stores. A budget that recorded the token + // it just refused would drift past its own cap. + let mut budget = budget(small()); + for _ in 0..3 { + budget.admit_token(AT).expect("within the token budget"); + } + budget.admit_token(AT).expect_err("the fourth token"); + budget.admit_token(AT).expect_err("the fifth token"); + assert_eq!(budget.tokens(), 3, "a refusal stores nothing"); + } + + #[test] + fn the_root_block_is_depth_one() { + let mut budget = budget(small()); + budget.enter_block(AT).expect("the score root"); + assert_eq!(budget.depth(), 1); + } + + #[test] + fn nesting_counts_simultaneously_open_blocks_not_total_blocks() { + // Two sibling blocks are depth 1 twice, never depth 2. A counter + // that never decremented would refuse a perfectly flat document. + let mut budget = budget(small()); + for _ in 0..10 { + budget.enter_block(AT).expect("a sibling block"); + budget.leave_block(); + } + assert_eq!(budget.depth(), 0); + } + + #[test] + fn the_block_that_would_exceed_the_depth_is_the_one_refused() { + let mut budget = budget(small()); + budget.enter_block(AT).expect("depth 1"); + budget.enter_block(AT).expect("depth 2"); + let refusal = budget.enter_block(AT).expect_err("depth 3 exceeds two"); + assert_eq!(refusal.code, "SWG0509"); + assert_eq!(budget.depth(), 2, "a refused block was never entered"); + } + + #[test] + fn diagnostic_exhaustion_is_terminal_and_stops_accounting() { + // `admit_token` already pins that a refused thing does not advance + // admitted state. The diagnostic axis needs the same law, adjusted + // for the one difference: its terminal refusal *does* consume the + // final slot, once. + // + // Without that, `diagnostics()` becomes false state the moment the + // budget goes terminal — it keeps climbing past its own cap on every + // later call, and the message invents an ever-larger hypothetical + // parse for a parse that has already been terminated. A caller + // obeying `Err` never sees it, which makes the bug latent, not + // correct: a type that calls itself a running resource state has to + // report one. + let cap = 2; + let mut budget = budget(small()); + budget.admit_diagnostic(AT).expect("the first diagnostic"); + for attempt in 0..6 { + let refusal = budget + .admit_diagnostic(AT) + .expect_err("the budget is terminal from the second on"); + assert_eq!(refusal.code, "SWG0509", "attempt {attempt}"); + assert_eq!( + budget.diagnostics(), + cap, + "attempt {attempt}: admitted state must saturate at the cap" + ); + assert!( + refusal.message.contains("needed 3"), + "attempt {attempt}: `needed` must stay at cap + 1, not grow: {}", + refusal.message + ); + } + } + + /// One breach per axis: the axis phrase, the declared limit, the count + /// the parse would have needed, and the refusal itself. + type Breach = (&'static str, u64, u64, Diagnostic); + + /// One breach per axis, each with the phrase, limit, needed count, and + /// span it must report. + fn every_breach() -> Vec { + let at = Span { start: 40, end: 44 }; + let source = budget(small()) + .admit_source("123456789", at) + .expect_err("nine bytes over a limit of eight"); + let mut token_budget = budget(small()); + for _ in 0..3 { + token_budget.admit_token(AT).expect("within the budget"); + } + let tokens = token_budget.admit_token(at).expect_err("the fourth token"); + let mut depth_budget = budget(small()); + for _ in 0..2 { + depth_budget.enter_block(AT).expect("within the budget"); + } + let depth = depth_budget.enter_block(at).expect_err("the third block"); + let mut diagnostic_budget = budget(small()); + diagnostic_budget + .admit_diagnostic(AT) + .expect("the first diagnostic"); + let diagnostics = diagnostic_budget + .admit_diagnostic(at) + .expect_err("the second leaves no room for the breach"); + vec![ + ("source bytes", 8, 9, source), + ("tokens", 3, 4, tokens), + ("nesting depth", 2, 3, depth), + // Two ordinary diagnostics plus the terminal refusal would need + // three slots against a cap of two, which is what `needed` says. + ("diagnostics", 2, 3, diagnostics), + ] + } + + #[test] + fn every_axis_reports_its_code_phrase_limit_needed_and_span() { + // One witness over all four axes. Checking the message on one axis + // and the span on another leaves the other six combinations free to + // rot: swapping `NestingDepth`'s word for `"tokens"`, or pointing a + // depth breach anywhere it liked, used to survive this suite. + let at = Span { start: 40, end: 44 }; + for (phrase, limit, needed, refusal) in every_breach() { + assert_eq!(refusal.code, "SWG0509", "{phrase}"); + assert!( + refusal.message.contains(phrase), + "{phrase} breach says: {}", + refusal.message + ); + let limit_text = format!("limit is {limit}"); + assert!( + refusal.message.contains(&limit_text), + "{phrase} breach must name its declared limit: {}", + refusal.message + ); + let needed_text = format!("needed {needed}"); + assert!( + refusal.message.contains(&needed_text), + "{phrase} breach must name what was needed: {}", + refusal.message + ); + assert_eq!(refusal.span, at, "{phrase} breach points at the caller"); + } + } + + #[test] + fn the_diagnostic_budget_reserves_its_last_slot_for_the_breach() { + // §5.11's diagnostic cap is on what one parse attempt *returns*, and + // the terminal resource diagnostic counts toward it. So a cap of two + // buys one ordinary diagnostic and the SWG0509 that ends the run — + // never two ordinary ones and a third that quietly exceeds the cap. + let mut budget = budget(small()); + budget.admit_diagnostic(AT).expect("the first diagnostic"); + let terminal = budget + .admit_diagnostic(AT) + .expect_err("the second would leave no room for the breach"); + assert_eq!(terminal.code, "SWG0509"); + assert_eq!(budget.diagnostics(), 2, "the terminal one is counted"); + } +} diff --git a/swang/tests/law_a_baseline.golden b/swang/tests/law_a_baseline.golden new file mode 100644 index 0000000..4cae361 --- /dev/null +++ b/swang/tests/law_a_baseline.golden @@ -0,0 +1,451 @@ +schema 1 +producer c44313c0cd82f3f2a8720437824d8cf5058b4e15 +cases 47 + +case fuzz_seed_reference +source 444 "swang 1\n\npattern dgd_fractal {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096 density 9500bps seed 4\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"corpus/Dance Gavin Dance - The Robot With Human Hair Part 2.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy repeat_variation\n corpus \"corpus\"\n }\n |> export midi \"dgd_fractal_dense.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 11 "dgd_fractal" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune present +ast fractalize.prune.density 9500 +ast fractalize.prune.seed 4 +ast linearize.traversal snake +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail rest_pad +ast generate.source 63 "corpus/Dance Gavin Dance - The Robot With Human Hair Part 2.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy named repeat_variation +ast generate.corpus present 6 "corpus" +ast export.format midi +ast export.path 21 "dgd_fractal_dense.mid" +canonical 444 "swang 1\n\npattern dgd_fractal {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096 density 9500bps seed 4\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"corpus/Dance Gavin Dance - The Robot With Human Hair Part 2.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy repeat_variation\n corpus \"corpus\"\n }\n |> export midi \"dgd_fractal_dense.mid\"\n}\n" +end + +case minimal_no_prune_no_corpus +source 306 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 1 "p" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune absent +ast linearize.traversal snake +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail rest_pad +ast generate.source 8 "seed.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy auto +ast generate.corpus absent +ast export.format midi +ast export.path 7 "out.mid" +canonical 306 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +end + +case prune_present +source 329 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096 density 9500bps seed 4\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 1 "p" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune present +ast fractalize.prune.density 9500 +ast fractalize.prune.seed 4 +ast linearize.traversal snake +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail rest_pad +ast generate.source 8 "seed.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy auto +ast generate.corpus absent +ast export.format midi +ast export.path 7 "out.mid" +canonical 329 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096 density 9500bps seed 4\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +end + +case row_major_and_reject_tail +source 308 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize row_major\n |> map_rhythm unit 1/16 tail reject\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 1 "p" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune absent +ast linearize.traversal row_major +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail reject +ast generate.source 8 "seed.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy auto +ast generate.corpus absent +ast export.format midi +ast export.path 7 "out.mid" +canonical 308 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize row_major\n |> map_rhythm unit 1/16 tail reject\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +end + +case words_reordered_off_canonical +source 306 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize max_cells 4096 depth 1\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 1 "p" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune absent +ast linearize.traversal snake +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail rest_pad +ast generate.source 8 "seed.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy auto +ast generate.corpus absent +ast export.format midi +ast export.path 7 "out.mid" +canonical 306 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +end + +case strategy_rhythm_copy +source 313 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy rhythm_copy\n }\n |> export midi \"out.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 1 "p" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune absent +ast linearize.traversal snake +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail rest_pad +ast generate.source 8 "seed.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy named rhythm_copy +ast generate.corpus absent +ast export.format midi +ast export.path 7 "out.mid" +canonical 313 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy rhythm_copy\n }\n |> export midi \"out.mid\"\n}\n" +end + +case strategy_motif_transpose +source 317 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy motif_transpose\n }\n |> export midi \"out.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 1 "p" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune absent +ast linearize.traversal snake +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail rest_pad +ast generate.source 8 "seed.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy named motif_transpose +ast generate.corpus absent +ast export.format midi +ast export.path 7 "out.mid" +canonical 317 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy motif_transpose\n }\n |> export midi \"out.mid\"\n}\n" +end + +case strategy_constrained_walk +source 318 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy constrained_walk\n }\n |> export midi \"out.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 1 "p" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune absent +ast linearize.traversal snake +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail rest_pad +ast generate.source 8 "seed.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy named constrained_walk +ast generate.corpus absent +ast export.format midi +ast export.path 7 "out.mid" +canonical 318 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy constrained_walk\n }\n |> export midi \"out.mid\"\n}\n" +end + +case strategy_shuffle_motifs +source 316 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy shuffle_motifs\n }\n |> export midi \"out.mid\"\n}\n" +verdict accepted +ast level 1 +ast pattern.name 1 "p" +ast pattern.kernel 11 "X.X/XX./.XX" +ast fractalize.depth 1 +ast fractalize.max_cells 4096 +ast fractalize.prune absent +ast linearize.traversal snake +ast map_rhythm.unit.numerator 1 +ast map_rhythm.unit.denominator 16 +ast map_rhythm.tail rest_pad +ast generate.source 8 "seed.gp5" +ast generate.bars 8 +ast generate.seed 42 +ast generate.candidates 2 +ast generate.strategy named shuffle_motifs +ast generate.corpus absent +ast export.format midi +ast export.path 7 "out.mid" +canonical 316 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy shuffle_motifs\n }\n |> export midi \"out.mid\"\n}\n" +end + +case kernel_whitespace +source 302 "swang 1\n\npattern p {\n ascii \"X X/XX.\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0103 31 40 69 "whitespace inside the kernel literal; rows are separated by `/` alone" +end + +case kernel_empty_row +source 302 "swang 1\n\npattern p {\n ascii \"X.X//XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0307 31 40 33 "empty kernel literal or empty row" +end + +case kernel_ragged +source 301 "swang 1\n\npattern p {\n ascii \"X.X/XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0101 31 39 44 "ragged kernel: row 1 has 2 cells, expected 3" +end + +case kernel_foreign_cell +source 300 "swang 1\n\npattern p {\n ascii \"XO/XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0102 31 38 57 "invalid kernel cell 'O' at row 0, col 1: only `X` and `.`" +end + +case unit_zero_numerator +source 306 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 0/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0301 132 136 25 "unit 0/16 has a zero part" +end + +case density_without_seed +source 322 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096 density 9500bps\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0303 86 93 80 "density decay was given without a rhythm seed; pruning must be explicitly seeded" +end + +case density_out_of_range +source 330 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096 density 10001bps seed 4\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0308 94 102 38 "density 10001 bps is outside 0..=10000" +end + +case unknown_traversal +source 307 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize spiral\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0402 103 109 56 "unknown traversal `spiral`; the set is row_major | snake" +end + +case missing_max_cells +source 291 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0403 52 62 53 "`fractalize` is missing its required word `max_cells`" +end + +case repeated_word +source 314 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 depth 2 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0404 71 76 44 "the word `depth` repeats within `fractalize`" +end + +case second_pattern_block +source 320 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\npattern q {\n}\n" +verdict rejected +diagnostic 0 SWG0401 306 313 53 "a program is one pattern block; nothing may follow it" +end + +case lex_bare_pipe +source 305 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n | linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 90 91 13 "expected `|>`" +end + +case lex_unexpected_character +source 307 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars @8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 208 209 24 "unexpected character '@'" +end + +case lex_unterminated_string +source 305 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 184 305 27 "unterminated string literal" +end + +case truncated_after_header +source 16 "swang 1\n\npattern" +verdict rejected +diagnostic 0 SWG0401 16 16 23 "unexpected end of input" +end + +case nothing_after_header +source 8 "swang 1\n" +verdict rejected +diagnostic 0 SWG0401 8 8 23 "unexpected end of input" +end + +case unclosed_pattern_block +source 304 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n" +verdict rejected +diagnostic 0 SWG0401 304 304 23 "unexpected end of input" +end + +case no_pattern_keyword +source 305 "swang 1\n\npatern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 9 15 34 "expected `pattern`, found `patern`" +end + +case no_pattern_name +source 304 "swang 1\n\npattern {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 17 18 34 "expected a pattern name, found `{`" +end + +case no_open_brace +source 304 "swang 1\n\npattern p\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 23 28 27 "expected `{`, found `ascii`" +end + +case first_element_not_ascii +source 307 "swang 1\n\npattern p {\n kernel \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0403 25 31 49 "the pattern block begins with its `ascii` literal" +end + +case ascii_value_not_a_string +source 296 "swang 1\n\npattern p {\n ascii 123\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 31 34 38 "expected a kernel literal, found `123`" +end + +case trailing_content_after_block +source 304 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 277 279 53 "a program is one pattern block; nothing may follow it" +end + +case unknown_pipeline_step +source 304 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> flatten snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 93 100 31 "unknown pipeline step `flatten`" +end + +case steps_out_of_order +source 306 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> linearize snake\n |> fractalize depth 1 max_cells 4096\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 52 61 64 "`linearize` arrives out of pipeline order; expected `fractalize`" +end + +case missing_pipeline_step +source 283 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0403 281 282 44 "the pipeline is missing its `linearize` step" +end + +case word_names_no_value +source 301 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 71 80 35 "the word `max_cells` names no value" +end + +case linearize_missing_its_word +source 300 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0403 93 102 52 "`linearize` is missing its required word `traversal`" +end + +case export_missing_its_path +source 296 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi\n}\n" +verdict rejected +diagnostic 0 SWG0401 282 288 34 "`export` takes a format and a path" +end + +case export_unknown_format +source 305 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export wav \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0402 289 292 44 "unknown export format `wav`; the set is midi" +end + +case integer_not_decimal +source 310 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars eight\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 208 213 49 "bars takes a plain decimal integer, found `eight`" +end + +case integer_leading_zero +source 307 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 08\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 208 210 32 "bars does not take leading zeros" +end + +case integer_out_of_range +source 331 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 99999999999999999999999999\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 208 234 55 "bars value `99999999999999999999999999` is out of range" +end + +case integer_too_wide_for_its_field +source 308 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 300 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 69 72 33 "depth value `300` is out of range" +end + +case density_without_the_bps_suffix +source 326 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096 density 9500 seed 4\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 94 98 78 "density takes basis points with the `bps` suffix, like `9500bps`; found `9500`" +end + +case unit_not_a_rational +source 304 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 16 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0301 132 134 52 "malformed unit `16`: expected a note value like 1/16" +end + +case unit_with_three_parts +source 308 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16/4 tail rest_pad\n |> generate {\n source \"seed.gp5\"\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0301 132 138 56 "malformed unit `1/16/4`: expected a note value like 1/16" +end + +case string_word_takes_no_bare_word +source 300 "swang 1\n\npattern p {\n ascii \"X.X/XX./.XX\"\n |> fractalize depth 1 max_cells 4096\n |> linearize snake\n |> map_rhythm unit 1/16 tail rest_pad\n |> generate {\n source seed\n bars 8\n seed 42\n candidates 2\n strategy auto\n }\n |> export midi \"out.mid\"\n}\n" +verdict rejected +diagnostic 0 SWG0401 184 188 28 "source takes a quoted string" +end diff --git a/swang/tests/law_a_baseline.rs b/swang/tests/law_a_baseline.rs new file mode 100644 index 0000000..2d23745 --- /dev/null +++ b/swang/tests/law_a_baseline.rs @@ -0,0 +1,966 @@ +//! SWG-INF-06: the frozen Law A baseline for language level 1. +//! +//! Spec §5.5 says a build supporting `1..=N` must treat every source whose +//! first line is a **valid `swang 1` header** — including those with +//! **invalid bodies** — exactly as a level-1-only build did, compared on +//! verdict, AST, canonical bytes, diagnostic code, message, span, and order. +//! The valid header is the premise, not part of what varies; the witness +//! below makes that domain executable rather than a claim in prose. Today `N == 1`, so that comparison has nothing on its +//! right-hand side; the moment SWG-4A-06 adds level dispatch it has +//! everything, and by then the level-1-only build is gone — exactly as the +//! pre-refactor parser SWG-INF-06's original sketch wanted to diff against +//! is already gone. +//! +//! So this suite records the left-hand side now, while a level-1-only build +//! is what the tree contains. The recorded document is a historical +//! artifact, not a re-derivation: it names the commit that produced it, and +//! it is **compare-only**. There is deliberately no "update the snapshot" +//! path — a diff here is a Law A violation until a human proves otherwise. +//! +//! Three deliberate properties of the observation: +//! +//! 1. **The AST observation is not the formatter's.** It is a test-owned +//! projection that reads accessors and spells every field and every enum +//! variant itself. Recording `format(ast)` twice would give canonical +//! bytes and the AST as one witness wearing two hats, and a coordinated +//! parser+formatter regression could preserve the bytes while changing +//! what the AST means. +//! 2. **It is not `Debug`, and it is not `serde`.** `Debug` output is a +//! representation detail no contract pins, and `Program` is deliberately +//! not serialized. Adding either to manufacture a witness would be +//! inventing a contract in order to test it. +//! 3. **The observables are outcome-dependent.** An accepted source has an +//! AST and canonical bytes; a rejected one has ordered diagnostics. No +//! null fields are invented so a schema can boast of holding seven +//! things. + +// Reason: integration-test code. `unwrap`/`expect`/`panic` abort loudly with +// a clear message, which is exactly what a test harness wants. +#![allow( + clippy::unwrap_used, + clippy::expect_used, + clippy::panic, + clippy::missing_assert_message +)] + +use std::fmt::{Arguments, Write as _}; + +use griff_pattern::{DensityBps, Traversal}; +use griff_swang::syntax::{ + format, header_level, parse, Diagnostic, Export, ExportFormat, Fractalize, Generate, Ident, + KernelLiteral, Level, Linearize, MapRhythm, PatternDef, Program, Prune, StrategyName, + StrategyPolicy, StringLiteral, Unit, +}; +use griff_swang::TailPolicy; + +/// The commit whose parser produced the recorded observations: a +/// level-1-only build with `LANGUAGE_LEVEL == 1`, one parser module, and no +/// level dispatch. +const PRODUCER: &str = "c44313c0cd82f3f2a8720437824d8cf5058b4e15"; + +/// The recorded document's schema. Bump it only when the encoding itself +/// changes, and only together with a regenerated baseline and a written +/// reason — never to make a failing comparison pass. +const SCHEMA: u32 = 1; + +/// The frozen artifact. `include_str!` so a missing or renamed baseline is a +/// compile error rather than a silently skipped test. +const BASELINE: &str = include_str!("law_a_baseline.golden"); + +/// The checked-in fuzz seed, included as an input subset of this corpus +/// rather than described from memory. +const FUZZ_SEED: &str = include_str!("../../fuzz/corpus/swang_parse/reference.swg"); + +// ── encoding ───────────────────────────────────────────────────────────── + +/// Appends one formatted line. Writing to a `String` cannot fail; the +/// `expect` documents that rather than hiding it. +fn put(out: &mut String, args: Arguments<'_>) { + out.write_fmt(args) + .expect("writing to a String cannot fail"); + out.push('\n'); +} + +/// Encodes a string as ` ""`. +/// +/// Length-prefixed so a truncation cannot hide, and escaped so every value +/// stays on one line. Iteration is over `char`s, not bytes: re-emitting a +/// multi-byte character byte by byte would corrupt it. +fn quoted(text: &str) -> String { + let mut body = String::from("\""); + for ch in text.chars() { + match ch { + '\\' => body.push_str("\\\\"), + '"' => body.push_str("\\\""), + '\n' => body.push_str("\\n"), + '\r' => body.push_str("\\r"), + '\t' => body.push_str("\\t"), + other if u32::from(other) < 0x20 || u32::from(other) == 0x7f => { + let escape = format!("\\x{:02x}", u32::from(other)); + body.push_str(&escape); + } + other => body.push(other), + } + } + body.push('"'); + format!("{} {body}", text.len()) +} + +// ── the AST observation ────────────────────────────────────────────────── +// +// Every struct below is destructured with no `..`, and every enum is matched +// with no wildcard arm. That is the exhaustiveness obligation: adding a +// field or a variant to the level-1 AST breaks this file at compile time, +// so the baseline can never silently stop observing part of the tree. + +/// The level, spelled from its accessor rather than its `Debug`. +fn observe_level(out: &mut String, level: Level) { + put(out, format_args!("ast level {}", level.get())); +} + +fn observe_program(out: &mut String, program: &Program) { + let Program { level, pattern } = program; + observe_level(out, *level); + observe_pattern(out, pattern); +} + +fn observe_pattern(out: &mut String, pattern: &PatternDef) { + let PatternDef { + name, + kernel, + fractalize, + linearize, + map_rhythm, + generate, + export, + } = pattern; + put( + out, + format_args!("ast pattern.name {}", quoted(name.as_str())), + ); + put( + out, + format_args!("ast pattern.kernel {}", quoted(kernel.as_str())), + ); + observe_fractalize(out, *fractalize); + observe_linearize(out, *linearize); + observe_map_rhythm(out, *map_rhythm); + observe_generate(out, generate); + observe_export(out, export); +} + +fn observe_fractalize(out: &mut String, fractalize: Fractalize) { + let Fractalize { + depth, + max_cells, + prune, + } = fractalize; + put(out, format_args!("ast fractalize.depth {depth}")); + put(out, format_args!("ast fractalize.max_cells {max_cells}")); + match prune { + None => put(out, format_args!("ast fractalize.prune absent")), + Some(Prune { density, seed }) => { + put(out, format_args!("ast fractalize.prune present")); + put( + out, + format_args!("ast fractalize.prune.density {}", density.get()), + ); + put(out, format_args!("ast fractalize.prune.seed {seed}")); + } + } +} + +fn observe_linearize(out: &mut String, linearize: Linearize) { + let Linearize { traversal } = linearize; + let spelled = match traversal { + Traversal::RowMajor => "row_major", + Traversal::Snake => "snake", + }; + put(out, format_args!("ast linearize.traversal {spelled}")); +} + +fn observe_map_rhythm(out: &mut String, map_rhythm: MapRhythm) { + let MapRhythm { unit, tail } = map_rhythm; + observe_unit(out, unit); + let spelled = match tail { + TailPolicy::Reject => "reject", + TailPolicy::RestPad => "rest_pad", + }; + put(out, format_args!("ast map_rhythm.tail {spelled}")); +} + +fn observe_unit(out: &mut String, unit: Unit) { + put( + out, + format_args!("ast map_rhythm.unit.numerator {}", unit.numerator()), + ); + put( + out, + format_args!("ast map_rhythm.unit.denominator {}", unit.denominator()), + ); +} + +fn observe_generate(out: &mut String, generate: &Generate) { + let Generate { + source, + bars, + seed, + candidates, + strategy, + corpus, + } = generate; + put( + out, + format_args!("ast generate.source {}", quoted(source.as_str())), + ); + put(out, format_args!("ast generate.bars {bars}")); + put(out, format_args!("ast generate.seed {seed}")); + put(out, format_args!("ast generate.candidates {candidates}")); + observe_strategy(out, *strategy); + match corpus { + None => put(out, format_args!("ast generate.corpus absent")), + Some(path) => put( + out, + format_args!("ast generate.corpus present {}", quoted(path.as_str())), + ), + } +} + +fn observe_strategy(out: &mut String, strategy: StrategyPolicy) { + match strategy { + StrategyPolicy::Auto => put(out, format_args!("ast generate.strategy auto")), + StrategyPolicy::Named(name) => { + let spelled = match name { + StrategyName::RhythmCopy => "rhythm_copy", + StrategyName::MotifTranspose => "motif_transpose", + StrategyName::ConstrainedWalk => "constrained_walk", + StrategyName::ShuffleMotifs => "shuffle_motifs", + StrategyName::RepeatVariation => "repeat_variation", + }; + put(out, format_args!("ast generate.strategy named {spelled}")); + } + } +} + +fn observe_export(out: &mut String, export: &Export) { + let Export { format: fmt, path } = export; + let spelled = match fmt { + ExportFormat::Midi => "midi", + }; + put(out, format_args!("ast export.format {spelled}")); + put( + out, + format_args!("ast export.path {}", quoted(path.as_str())), + ); +} + +/// The diagnostic observation: code, span, and message, in the order the +/// parser returned them. Order is itself an observable (spec §5.5). +fn observe_diagnostic(out: &mut String, index: usize, d: &Diagnostic) { + let Diagnostic { + code, + span, + message, + } = d; + put( + out, + format_args!( + "diagnostic {index} {code} {} {} {}", + span.start, + span.end, + quoted(message) + ), + ); +} + +// ── the corpus ─────────────────────────────────────────────────────────── +// +// A deliberate Law A corpus, not a fuzz museum: every case is a fixed, +// deterministic source, and between them they exercise both verdicts, every +// level-1 enum variant, both states of every optional, and every diagnostic +// code level 1 can emit. The checked-in fuzz seed is included as an input +// subset rather than stood in for. + +/// One recorded case: a stable name and the exact bytes fed to the parser. +struct Case { + name: &'static str, + source: String, +} + +/// A level-1 script assembled from replaceable parts, so a case can vary one +/// construct without restating the program. +#[derive(Clone, Copy)] +struct Script { + kernel: &'static str, + fractalize: &'static str, + linearize: &'static str, + map_rhythm: &'static str, + generate: &'static str, + export: &'static str, +} + +/// The `generate` body every case shares but the strategy cases. +const GENERATE_AUTO: &str = " source \"seed.gp5\"\n bars 8\n \ + seed 42\n candidates 2\n strategy auto"; + +impl Script { + const fn base() -> Self { + Self { + kernel: "X.X/XX./.XX", + fractalize: "depth 1 max_cells 4096", + linearize: "snake", + map_rhythm: "unit 1/16 tail rest_pad", + generate: GENERATE_AUTO, + export: "midi \"out.mid\"", + } + } + + fn render(self) -> String { + let Self { + kernel, + fractalize, + linearize, + map_rhythm, + generate, + export, + } = self; + format!( + "swang 1\n\npattern p {{\n ascii \"{kernel}\"\n \ + |> fractalize {fractalize}\n |> linearize {linearize}\n \ + |> map_rhythm {map_rhythm}\n |> generate {{\n{generate}\n }}\n \ + |> export {export}\n}}\n" + ) + } +} + +/// A case built from a script. +fn case(name: &'static str, script: Script) -> Case { + Case { + name, + source: script.render(), + } +} + +/// A case whose only departure from the base script is the named strategy. +fn strategy_case(name: &'static str, generate: &'static str) -> Case { + case( + name, + Script { + generate, + ..Script::base() + }, + ) +} + +const GEN_RHYTHM_COPY: &str = " source \"seed.gp5\"\n bars 8\n \ + seed 42\n candidates 2\n strategy rhythm_copy"; +const GEN_MOTIF_TRANSPOSE: &str = " source \"seed.gp5\"\n bars 8\n \ + seed 42\n candidates 2\n strategy motif_transpose"; +const GEN_CONSTRAINED_WALK: &str = " source \"seed.gp5\"\n bars 8\n \ + seed 42\n candidates 2\n strategy constrained_walk"; +const GEN_SHUFFLE_MOTIFS: &str = " source \"seed.gp5\"\n bars 8\n \ + seed 42\n candidates 2\n strategy shuffle_motifs"; + +/// The sources level 1 accepts. +fn accepted_corpus() -> Vec { + let base = Script::base(); + vec![ + Case { + name: "fuzz_seed_reference", + source: FUZZ_SEED.to_owned(), + }, + case("minimal_no_prune_no_corpus", base), + case( + "prune_present", + Script { + fractalize: "depth 1 max_cells 4096 density 9500bps seed 4", + ..base + }, + ), + case( + "row_major_and_reject_tail", + Script { + linearize: "row_major", + map_rhythm: "unit 1/16 tail reject", + ..base + }, + ), + case( + "words_reordered_off_canonical", + Script { + fractalize: "max_cells 4096 depth 1", + ..base + }, + ), + strategy_case("strategy_rhythm_copy", GEN_RHYTHM_COPY), + strategy_case("strategy_motif_transpose", GEN_MOTIF_TRANSPOSE), + strategy_case("strategy_constrained_walk", GEN_CONSTRAINED_WALK), + strategy_case("strategy_shuffle_motifs", GEN_SHUFFLE_MOTIFS), + ] +} + +/// The kernel literal's own registry laws, in the transport's order. +fn rejected_kernel_corpus() -> Vec { + let base = Script::base(); + vec![ + case( + "kernel_whitespace", + Script { + kernel: "X X/XX.", + ..base + }, + ), + case( + "kernel_empty_row", + Script { + kernel: "X.X//XX", + ..base + }, + ), + case( + "kernel_ragged", + Script { + kernel: "X.X/XX", + ..base + }, + ), + case( + "kernel_foreign_cell", + Script { + kernel: "XO/XX", + ..base + }, + ), + ] +} + +/// The word-level refusals: unknown, missing, repeated, out of range, and +/// the structural catch-all. +fn rejected_word_corpus() -> Vec { + let base = Script::base(); + let second_pattern = format!("{}\npattern q {{\n}}\n", base.render().trim_end()); + vec![ + case( + "unit_zero_numerator", + Script { + map_rhythm: "unit 0/16 tail rest_pad", + ..base + }, + ), + case( + "density_without_seed", + Script { + fractalize: "depth 1 max_cells 4096 density 9500bps", + ..base + }, + ), + case( + "density_out_of_range", + Script { + fractalize: "depth 1 max_cells 4096 density 10001bps seed 4", + ..base + }, + ), + case( + "unknown_traversal", + Script { + linearize: "spiral", + ..base + }, + ), + case( + "missing_max_cells", + Script { + fractalize: "depth 1", + ..base + }, + ), + case( + "repeated_word", + Script { + fractalize: "depth 1 depth 2 max_cells 4096", + ..base + }, + ), + Case { + name: "second_pattern_block", + source: second_pattern, + }, + ] +} + +/// One case built by editing the base script's rendered text. +fn edited(name: &'static str, from: &str, to: &str) -> Case { + let source = Script::base().render(); + assert!( + source.contains(from), + "{name}: `{from}` is not in the base script" + ); + Case { + name, + source: source.replace(from, to), + } +} + +/// The lexer's own three refusals, and the shapes that never reach a word. +fn rejected_lexical_corpus() -> Vec { + vec![ + edited("lex_bare_pipe", "|> linearize", "| linearize"), + edited("lex_unexpected_character", "bars 8", "bars @8"), + edited("lex_unterminated_string", "\"seed.gp5\"", "\"seed.gp5"), + Case { + name: "truncated_after_header", + source: "swang 1\n\npattern".to_owned(), + }, + Case { + name: "nothing_after_header", + source: "swang 1\n".to_owned(), + }, + Case { + name: "unclosed_pattern_block", + source: { + let full = Script::base().render(); + full.trim_end().trim_end_matches('}').to_owned() + }, + }, + ] +} + +/// The block's own structure: the keyword, the name, the braces, and the +/// `ascii` literal that must come first. +fn rejected_block_corpus() -> Vec { + vec![ + edited("no_pattern_keyword", "pattern p {", "patern p {"), + edited("no_pattern_name", "pattern p {", "pattern {"), + edited("no_open_brace", "pattern p {", "pattern p"), + edited( + "first_element_not_ascii", + "ascii \"X.X/XX./.XX\"", + "kernel \"X.X/XX./.XX\"", + ), + edited( + "ascii_value_not_a_string", + "ascii \"X.X/XX./.XX\"", + "ascii 123", + ), + edited( + "trailing_content_after_block", + "|> generate {", + "|> generate", + ), + ] +} + +/// The pipeline's shape: which steps, in which order, with which words. +fn rejected_pipeline_corpus() -> Vec { + vec![ + edited( + "unknown_pipeline_step", + "|> linearize snake", + "|> flatten snake", + ), + edited( + "steps_out_of_order", + "|> fractalize depth 1 max_cells 4096\n |> linearize snake", + "|> linearize snake\n |> fractalize depth 1 max_cells 4096", + ), + edited("missing_pipeline_step", " |> linearize snake\n", ""), + edited( + "word_names_no_value", + "depth 1 max_cells 4096", + "depth 1 max_cells", + ), + edited( + "linearize_missing_its_word", + "|> linearize snake", + "|> linearize", + ), + edited( + "export_missing_its_path", + "export midi \"out.mid\"", + "export midi", + ), + edited("export_unknown_format", "export midi", "export wav"), + ] +} + +/// The scalar spellings: integers, densities, units, and quoted strings. +fn rejected_scalar_corpus() -> Vec { + vec![ + edited("integer_not_decimal", "bars 8", "bars eight"), + edited("integer_leading_zero", "bars 8", "bars 08"), + edited( + "integer_out_of_range", + "bars 8", + "bars 99999999999999999999999999", + ), + edited("integer_too_wide_for_its_field", "depth 1", "depth 300"), + edited( + "density_without_the_bps_suffix", + "depth 1 max_cells 4096", + "depth 1 max_cells 4096 density 9500 seed 4", + ), + edited("unit_not_a_rational", "unit 1/16", "unit 16"), + edited("unit_with_three_parts", "unit 1/16", "unit 1/16/4"), + edited( + "string_word_takes_no_bare_word", + "source \"seed.gp5\"", + "source seed", + ), + ] +} + +/// The whole corpus, in the order the baseline records it. +fn corpus() -> Vec { + let mut all = accepted_corpus(); + all.extend(rejected_kernel_corpus()); + all.extend(rejected_word_corpus()); + all.extend(rejected_lexical_corpus()); + all.extend(rejected_block_corpus()); + all.extend(rejected_pipeline_corpus()); + all.extend(rejected_scalar_corpus()); + all +} + +// ── the recorded document ──────────────────────────────────────────────── + +/// Renders the observation document for the whole corpus. This is the only +/// producer of the baseline's content, and nothing in the repository writes +/// its output back to disk: the artifact is regenerated by hand, under +/// review, or not at all. +pub(crate) fn render_document() -> String { + let cases = corpus(); + let mut out = String::new(); + put(&mut out, format_args!("schema {SCHEMA}")); + put(&mut out, format_args!("producer {PRODUCER}")); + put(&mut out, format_args!("cases {}", cases.len())); + for entry in &cases { + out.push('\n'); + put(&mut out, format_args!("case {}", entry.name)); + put(&mut out, format_args!("source {}", quoted(&entry.source))); + observe_outcome(&mut out, &entry.source); + put(&mut out, format_args!("end")); + } + out +} + +/// Observes one source. The observables are outcome-dependent: an accepted +/// source has an AST and canonical bytes, a rejected one has ordered +/// diagnostics, and neither borrows a null-filled field from the other. +fn observe_outcome(out: &mut String, source: &str) { + match parse(source) { + Ok(program) => { + put(out, format_args!("verdict accepted")); + observe_program(out, &program); + let canonical = format(&program); + put(out, format_args!("canonical {}", quoted(&canonical))); + } + Err(diagnostics) => { + put(out, format_args!("verdict rejected")); + for (index, diagnostic) in diagnostics.iter().enumerate() { + observe_diagnostic(out, index, diagnostic); + } + } + } +} + +/// The AST observation alone, for the witnesses that compare two trees. +fn observed(program: &Program) -> String { + let mut out = String::new(); + observe_program(&mut out, program); + out +} + +// ── the tests ──────────────────────────────────────────────────────────── + +/// Every diagnostic code reachable **inside Law A's domain** — that is, from +/// a source whose first line is already a valid `swang 1` header. +/// +/// The header pre-parser's own three codes are deliberately absent. +/// `SWG0001`, `SWG0002`, and `SWG0003` are raised *instead of* admitting a +/// level-1 source, so no source that reaches them is a Law A case, and a +/// build supporting `1..=N` is under no obligation to keep raising them — +/// `SWG0001` for `swang 2` is precisely what SWG-4A-06 must stop doing. They +/// keep their own characterization tests beside the frozen pre-parser, where +/// the contract that governs them lives. +const LEVEL_ONE_CODES: &[&str] = &[ + "SWG0101", "SWG0102", "SWG0103", "SWG0301", "SWG0303", "SWG0307", "SWG0308", "SWG0401", + "SWG0402", "SWG0403", "SWG0404", +]; + +#[test] +fn every_corpus_source_is_inside_law_a_s_domain() { + // §5.5 states Law A over "every source whose first line is a valid + // `swang 1` header — whether its body is valid or not". The valid header + // is the premise, not part of what varies: only the body may be invalid. + // + // A source the frozen pre-parser refuses is therefore not a Law A case, + // and recording one here does active harm. `swang 2` is the proof. Its + // refusal today is `SWG0001`, and SWG-4A-06 exists precisely to make + // `swang 2` supported — after which it must pass header dispatch and be + // refused as a level-2 `pattern` root instead. A baseline built to + // protect 4A-06 would then declare 4A-06's whole purpose a regression. + // + // So the domain is a witness, not a convention. A malformed header, a + // BOM, and a newer level are characterized by the header suite that owns + // the frozen pre-parser; they have no business in this artifact. + for entry in &corpus() { + assert_eq!( + header_level(&entry.source), + Ok(1), + "{} is not a Law A source: §5.5's domain is sources whose first \ + line is a valid `swang 1` header", + entry.name + ); + } +} + +#[test] +fn the_recorded_baseline_is_what_this_build_still_produces() { + assert_eq!( + render_document(), + BASELINE, + "Law A: a level-1 source's verdict, AST, canonical bytes, and \ + diagnostics must not move. Regenerating this artifact is not the \ + fix — the parser changed, or the corpus did." + ); +} + +#[test] +fn the_baseline_names_the_build_that_produced_it() { + let mut lines = BASELINE.lines(); + assert_eq!(lines.next(), Some(format!("schema {SCHEMA}").as_str())); + assert_eq!(lines.next(), Some(format!("producer {PRODUCER}").as_str())); +} + +/// Distinct `(code, message)` refusals the corpus reaches. Pinned so the +/// sample cannot silently shrink. +const DISTINCT_REFUSALS: usize = 35; + +#[test] +fn the_corpus_pins_the_extent_of_its_own_sample() { + // Coverage by code is weaker than coverage by production site, and the + // difference is not academic: `SWG0403` is raised from four different + // places, and a corpus reaching one of them proves nothing about the + // other three. A falsification probe that corrupted an unreached site + // survived this suite until the corpus grew to reach it. + // + // No finite corpus can be Law A's whole domain. What it can do is state + // its own extent, so a shrinking sample is a failure rather than a + // quieter test run. + let mut pairs: Vec<(&str, String)> = Vec::new(); + for entry in &corpus() { + if let Err(diagnostics) = parse(&entry.source) { + for diagnostic in &diagnostics { + let pair = (diagnostic.code, diagnostic.message.clone()); + if !pairs.contains(&pair) { + pairs.push(pair); + } + } + } + } + assert_eq!( + pairs.len(), + DISTINCT_REFUSALS, + "the corpus reaches a different set of refusals than it records" + ); +} + +#[test] +fn the_corpus_reaches_every_level_one_diagnostic_code() { + let mut seen: Vec<&str> = Vec::new(); + for entry in &corpus() { + if let Err(diagnostics) = parse(&entry.source) { + for diagnostic in &diagnostics { + if !seen.contains(&diagnostic.code) { + seen.push(diagnostic.code); + } + } + } + } + for code in LEVEL_ONE_CODES { + assert!( + seen.contains(code), + "no corpus case reaches {code}; Law A would go unwitnessed there" + ); + } + for code in &seen { + assert!( + LEVEL_ONE_CODES.contains(code), + "{code} is not in the frozen level-1 registry list" + ); + } +} + +#[test] +fn the_corpus_records_both_verdicts_and_names_every_case_once() { + let all = corpus(); + let accepted = all.iter().filter(|c| parse(&c.source).is_ok()).count(); + assert_eq!(accepted, accepted_corpus().len()); + assert!(accepted < all.len(), "the corpus must record refusals too"); + let mut names: Vec<&str> = all.iter().map(|c| c.name).collect(); + names.sort_unstable(); + let before = names.len(); + names.dedup(); + assert_eq!(names.len(), before, "case names are the artifact's keys"); +} + +/// One leaf mutation: a name for the failure message, and the edit. +type Mutation = (&'static str, fn(&mut Program)); + +/// Every leaf of the base script's AST that has a second inhabitant. +/// +/// Two observed leaves are deliberately absent, because no mutation of them +/// exists to write: `level` accepts only `1` on this build, and +/// `ExportFormat` has exactly one variant. Both are still observed, and both +/// gain a mutation the moment they gain an inhabitant — the exhaustive +/// `match` in the projection will not compile until they do. +fn structural_mutations() -> Vec { + vec![ + ("pattern.name", |p| { + p.pattern.name = Ident::new("q").expect("an identifier"); + }), + ("pattern.kernel", |p| { + p.pattern.kernel = KernelLiteral::new("XX/XX").expect("a kernel"); + }), + ("fractalize.depth", |p| p.pattern.fractalize.depth = 2), + ("fractalize.max_cells", |p| { + p.pattern.fractalize.max_cells = 4097; + }), + ("fractalize.prune presence", |p| { + p.pattern.fractalize.prune = Some(Prune { + density: DensityBps::new(1).expect("in scale"), + seed: 0, + }); + }), + ("linearize.traversal", |p| { + p.pattern.linearize.traversal = Traversal::RowMajor; + }), + ("map_rhythm.unit.numerator", |p| { + p.pattern.map_rhythm.unit = Unit::new(2, 16).expect("a unit"); + }), + ("map_rhythm.unit.denominator", |p| { + p.pattern.map_rhythm.unit = Unit::new(1, 8).expect("a unit"); + }), + ("map_rhythm.tail", |p| { + p.pattern.map_rhythm.tail = TailPolicy::Reject; + }), + ] +} + +/// The `generate` and `export` leaves. +fn edge_mutations() -> Vec { + vec![ + ("generate.source", |p| { + p.pattern.generate.source = StringLiteral::new("other.gp5").expect("a path"); + }), + ("generate.bars", |p| p.pattern.generate.bars = 9), + ("generate.seed", |p| p.pattern.generate.seed = 43), + ("generate.candidates", |p| p.pattern.generate.candidates = 3), + ("generate.strategy", |p| { + p.pattern.generate.strategy = StrategyPolicy::Named(StrategyName::RepeatVariation); + }), + ("generate.corpus presence", |p| { + p.pattern.generate.corpus = Some(StringLiteral::new("corpus").expect("a path")); + }), + ("export.path", |p| { + p.pattern.export.path = StringLiteral::new("other.mid").expect("a path"); + }), + ] +} + +/// The two leaves that only exist once pruning is present. +fn pruning_mutations() -> Vec { + vec![ + ("prune.density", |p| { + if let Some(prune) = p.pattern.fractalize.prune.as_mut() { + prune.density = DensityBps::new(1).expect("in scale"); + } + }), + ("prune.seed", |p| { + if let Some(prune) = p.pattern.fractalize.prune.as_mut() { + prune.seed = 5; + } + }), + ] +} + +/// Proves each mutation actually moves the recorded observation. +fn assert_every_leaf_moves(base: &Program, mutations: Vec) { + let baseline = observed(base); + for (leaf, mutate) in mutations { + let mut mutated = base.clone(); + mutate(&mut mutated); + assert_ne!( + observed(&mutated), + baseline, + "changing {leaf} left the AST observation identical, so the \ + baseline is not watching that leaf" + ); + } +} + +#[test] +fn the_ast_observation_notices_every_leaf_with_a_second_inhabitant() { + // The projection is what makes the AST an *independent* witness from the + // canonical bytes. A leaf it fails to move is a leaf a coordinated + // parser and formatter change could rewrite while both recorded + // observables stayed still. + let base = parse(&Script::base().render()).expect("the base script parses"); + assert_every_leaf_moves(&base, structural_mutations()); + assert_every_leaf_moves(&base, edge_mutations()); +} + +#[test] +fn the_ast_observation_notices_both_pruning_leaves() { + let script = Script { + fractalize: "depth 1 max_cells 4096 density 9500bps seed 4", + ..Script::base() + }; + let base = parse(&script.render()).expect("the pruning script parses"); + assert!(base.pattern.fractalize.prune.is_some()); + assert_every_leaf_moves(&base, pruning_mutations()); +} + +#[test] +fn a_source_off_the_canonical_form_still_records_canonical_bytes() { + // `words_reordered_off_canonical` exists so the recorded canonical bytes + // are provably not a copy of the source: if they were, this case would + // record its own off-canonical spelling. + let script = Script { + fractalize: "max_cells 4096 depth 1", + ..Script::base() + }; + let source = script.render(); + let program = parse(&source).expect("word order is free within a construct"); + let canonical = format(&program); + assert_ne!(canonical, source); + assert!(canonical.contains("depth 1 max_cells 4096")); +} + +#[test] +fn every_recorded_code_has_the_one_registry_shape() { + // The same law the `swang_parse` fuzz oracle asserts, kept where + // `cargo test` runs it too: the registry has one shape, `SWG` and four + // digits. `starts_with("SWG")` would accept `SWG`, `SWGxyz`, and + // `SWG12345` as registry codes. + let shaped = |code: &str| { + code.strip_prefix("SWG") + .is_some_and(|rest| rest.len() == 4 && rest.bytes().all(|b| b.is_ascii_digit())) + }; + for code in LEVEL_ONE_CODES { + assert!(shaped(code), "{code} is not SWG followed by four digits"); + } + for entry in &corpus() { + if let Err(diagnostics) = parse(&entry.source) { + for diagnostic in &diagnostics { + assert!( + shaped(diagnostic.code), + "{} in {}", + diagnostic.code, + entry.name + ); + } + } + } +} diff --git a/swang/tests/level_two_budget_boundary.rs b/swang/tests/level_two_budget_boundary.rs new file mode 100644 index 0000000..cc34c83 --- /dev/null +++ b/swang/tests/level_two_budget_boundary.rs @@ -0,0 +1,375 @@ +//! SWG-INF-06: the level-2 budget stays off every level-1 path. +//! +//! Spec §5.11 allows input bounds at level 2 only, because level 1's +//! acceptance set is frozen (§5.5): a bound that rejected a source level 1 +//! accepts would be an observable change to a frozen level. That makes "no +//! level-1 path consults the budget" a contract, not a coding preference, +//! and a contract nobody checks is a comment. +//! +//! These witnesses read the shipped source rather than the behaviour, +//! because the failure they guard against is a *call site*, and by the time +//! behaviour shows it the frozen level has already moved. Comment-only lines +//! are stripped first: this file's own subject matter is discussed in the +//! prose of the very modules it scans, and a witness that reads prose as +//! code is a witness that fires on a docstring. + +// Reason: integration-test code. `unwrap`/`expect`/`panic` abort loudly with +// a clear message, which is exactly what a test harness wants. +#![allow( + clippy::unwrap_used, + clippy::expect_used, + clippy::panic, + clippy::missing_assert_message +)] + +use std::ffi::OsStr; +use std::fs::{read_dir, read_to_string}; +use std::path::{Component, Path, PathBuf}; + +/// The level-1 parse path, end to end: the frozen header pre-parser, the +/// lexer it hands off to, the one parser module, and the formatter. +const LEVEL_ONE_PATH: &[(&str, &str)] = &[ + // `syntax.rs` is the crate's public re-export point and the natural home + // for shared dispatch, which makes it the one file most worth scanning — + // a budget consulted there, *before* the level 1/2 branch, would be + // consulted on every level-1 parse while both of this suite's other + // witnesses stayed silent. + ("syntax.rs", include_str!("../src/syntax.rs")), + ("header.rs", include_str!("../src/syntax/header.rs")), + ("lexer.rs", include_str!("../src/syntax/lexer.rs")), + ("parser/v1.rs", include_str!("../src/syntax/parser/v1.rs")), + ("format/v1.rs", include_str!("../src/syntax/format/v1.rs")), + ("ast/v1.rs", include_str!("../src/syntax/ast/v1.rs")), + ("eval.rs", include_str!("../src/eval.rs")), +]; + +/// Every name the budget exports. A level-1 module naming any of them is +/// consulting a level-2 bound. +const BUDGET_NAMES: &[&str] = &[ + // Qualified, because bare `limits` is an ordinary English word and an + // ordinary Rust identifier. Every real route to the module — `use + // crate::syntax::limits::…`, `super::limits::…`, an inline + // `limits::MAX_TOKENS` — carries the `::`; prose and unrelated locals do + // not. + "limits::", + "Level2Budget", + "Level2ResourceLimits", + "MAX_SOURCE_BYTES", + "MAX_TOKENS", + "MAX_NESTING_DEPTH", + "MAX_DIAGNOSTICS", + "SWG0509", + // Bare method names are still absent — they are ordinary identifiers, + // and banning them failed CI on an unrelated `fn enter_block(...)`. + // What is banned is the *call*, which is punctuated: a definition, a + // binding, or a sentence never contains `.name(`. + // + // This closes the route the earlier form left open — a helper hands + // back a `Level2Budget`, type inference supplies the type, and the call + // site spells no name at all. Matching the call needs neither the type + // nor a Rust parser. + // + // rustfmt breaks a long chain before the dot, so `.name(` stays + // contiguous even across lines; the repository's own formatting is what + // makes a plain substring enough. + ".admit_source(", + ".admit_token(", + ".enter_block(", + ".leave_block(", + ".admit_diagnostic(", +]; + +/// The one line `syntax.rs` may contain: the module declaration itself. +const DECLARATION: &str = "mod limits;"; + +/// Strips comment-only lines, so prose about a name is not read as a use of +/// it, and — in `syntax.rs` alone — the bare module declaration. +/// +/// Exempting `syntax.rs` wholesale was the earlier form, and it was wrong in +/// a way that only shows up next task: the file that declares the module is +/// also the file where level dispatch will live, so a blanket exemption +/// makes the shared dispatch point the one place a budget could be consulted +/// unwatched. The declaration is permitted by exact line; every other line +/// is scanned like any other module's. +fn code_of(source: &str) -> String { + strip(source, false) +} + +/// `code_of`, optionally also dropping the module declaration. +fn strip(source: &str, is_root: bool) -> String { + source + .lines() + .filter(|line| { + let trimmed = line.trim_start(); + !(trimmed.starts_with("//") || is_root && trimmed.trim_end() == DECLARATION) + }) + .collect::>() + .join("\n") +} + +/// Whether this path is the crate's `syntax.rs` root module. +fn is_root(shown: &str) -> bool { + shown.trim_start_matches('/') == "syntax.rs" +} + +/// Whether `haystack` mentions `needle` as a whole token, so a longer +/// identifier containing it does not count as a mention. +/// +/// A boundary is required only on the sides where the needle's own edge is a +/// word character. `limits::` ends in punctuation and is followed by the +/// name it qualifies, so demanding a non-word character after it would match +/// nothing at all. +fn mentions(haystack: &str, needle: &str) -> bool { + let bytes = haystack.as_bytes(); + let is_word = |b: Option<&u8>| b.is_some_and(|&b| b.is_ascii_alphanumeric() || b == b'_'); + let edge_is_word = |b: Option<&u8>| b.is_some_and(|&b| b.is_ascii_alphanumeric() || b == b'_'); + let needs_before = edge_is_word(needle.as_bytes().first()); + let needs_after = edge_is_word(needle.as_bytes().last()); + haystack.match_indices(needle).any(|(at, _)| { + let before = at.checked_sub(1).and_then(|i| bytes.get(i)); + let after = bytes.get(at.saturating_add(needle.len())); + !(needs_before && is_word(before) || needs_after && is_word(after)) + }) +} + +/// Whether a source names any budget identifier, after stripping the lines a +/// scan must not read. +fn names_the_budget(source: &str) -> bool { + let code = code_of(source); + BUDGET_NAMES.iter().any(|name| mentions(&code, name)) +} + +#[test] +fn the_exemption_is_decided_by_path_components_not_by_a_slashed_string() { + // `to_string_lossy` performs no separator normalisation, so a path built + // on Windows stringifies as `syntax\limits.rs` and never equals the + // literal `"syntax/limits.rs"`. The exempt budget module then stops + // being exempt and the guard fails on the one file it must ignore — + // which reads like a boundary breach and is not one. + // + // CI is `ubuntu-latest` on every job, so this is latent rather than + // live. It is still wrong on a platform this crate builds for, and the + // cure is to compare paths as paths rather than to translate separators + // by hand. + let native = Path::new("syntax").join("limits.rs"); + assert!( + is_exempt(&native), + "the budget module is exempt on any platform" + ); + let also = Path::new("syntax").join("tests.rs"); + assert!(is_exempt(&also), "so are its tests"); + assert!( + !is_exempt(Path::new("syntax").join("parser").join("v1.rs").as_path()), + "nothing else is" + ); +} + +#[test] +fn a_generic_word_in_prose_or_a_string_is_not_a_budget_reference() { + // This witness reads source text, not a parsed tree, so its precision is + // lexical and heuristic — never syntactic. That is an accepted trade: + // pulling in a Rust parser to guard a boundary test would cost more than + // the boundary is worth. + // + // What the trade must not buy is a guard that trips on English. `limits` + // is an ordinary word and an ordinary identifier, and a check that fails + // CI because someone wrote "no limits apply" in a string is a check the + // next person weakens — at which point the frozen level loses its guard + // for a reason that had nothing to do with the frozen level. + for benign in [ + r#"const _NOTE: &str = "no limits apply here";"#, + "const _N: u8 = 1; // nothing to do with limits", + "let limits = compute_ui_limits();", + "//! This module documents the limits elsewhere.", + "struct Delimiters;", + // A bare method name is an ordinary identifier and is not the + // signal — banning one failed CI on an unrelated definition. The + // call form is watched instead, which is why a definition and a + // binding still read as benign here while `b.enter_block(at)` does + // not. + "fn enter_block(&mut self) -> bool { true }", + "let admit_source = compute();", + ] { + assert!( + !names_the_budget(benign), + "benign source must not count as a budget reference: {benign}" + ); + } + + // And what it must still buy is every real way to reach the module. + for real in [ + "const _P: bool = limits::MAX_TOKENS > 0;", + "use crate::syntax::limits::Level2Budget;", + "use super::limits::Level2ResourceLimits;", + "let b = Level2Budget::declared();", + "if bytes > MAX_SOURCE_BYTES { refuse() }", + ] { + assert!( + names_the_budget(real), + "a real budget reference must count: {real}" + ); + } +} + +#[test] +fn a_budget_call_through_an_inferred_receiver_is_a_budget_reference() { + // Matching names alone leaves one route open, and review named it + // exactly: a helper hands back a `Level2Budget`, type inference supplies + // the type, and the call site never spells the module, the type, or a + // constant. + // + // let mut b = budget_from_context(); + // b.admit_token(at)?; + // + // No budget name appears, so the guard stayed silent while a level-1 + // module spent a level-2 bound. That is the failure this file exists to + // refuse, reached by the one path it did not watch. + // + // The cure is not a parser. A *call* carries punctuation — `.name(` — + // and a definition, a binding, or a sentence does not. That is why the + // bare identifiers were removed and these are safe to add: the marker + // that matched `fn enter_block(...)` is not the marker that matches + // `b.enter_block(at)`. + for call in [ + "let mut b = budget_from_context();\n b.admit_source(src, at)?;", + "let mut b = budget_from_context();\n b.admit_token(at)?;", + "b.enter_block(at)?;", + "b.leave_block();", + "self.budget.admit_diagnostic(at)?;", + // rustfmt breaks a long chain before the dot, which keeps `.name(` + // contiguous on the continuation line. The repository's own + // formatting is therefore what makes a plain substring sufficient, + // and no tolerance for optional whitespace is needed. + "some_budget\n .admit_token(at)\n .map_err(one)?;", + ] { + assert!( + names_the_budget(call), + "a budget call through an inferred receiver must count: {call}" + ); + } + + // And the bare identifiers must stay unmatched, because being noisy is + // why they were dropped. A call marker that also fired on these would + // have re-imported the false positives it was meant to shed. + for benign in [ + "fn admit_token(&mut self, at: Span) -> bool { true }", + "fn enter_block(&mut self) -> bool { true }", + "let enter_block = compute();", + "let admit_source = compute();", + r#"const _NOTE: &str = "admit_token is explained below";"#, + ] { + assert!( + !names_the_budget(benign), + "a bare method identifier must not count: {benign}" + ); + } +} + +#[test] +fn no_level_one_module_consults_the_level_two_budget() { + for (name, source) in LEVEL_ONE_PATH { + let code = strip(source, is_root(name)); + for budget_name in BUDGET_NAMES { + assert!( + !mentions(&code, budget_name), + "{name} names `{budget_name}`. Level 1's acceptance set is \ + frozen (§5.5); a declared bound reaching it is a narrowing \ + of a frozen level, not an optimisation." + ); + } + } +} + +#[test] +fn the_witness_can_fail() { + // A boundary test that cannot fail proves nothing about the boundary. If + // `mentions` or `code_of` ever stopped seeing real code, the witness + // above would pass for the wrong reason and no one would learn of it. + let planted = "fn lex() { let b = Level2Budget::new(limits::declared()); }"; + assert!(names_the_budget(planted)); + assert!(mentions(&code_of(planted), "Level2Budget")); + assert!(mentions(&code_of(planted), "limits::")); + let prose = "//! The Level2Budget is discussed here but never called."; + assert!(!mentions(&code_of(prose), "Level2Budget")); + // The declaration is permitted in the root module and nowhere else, and + // permitting it must not swallow a use on the same subject. + assert!(!mentions(&strip("mod limits;", true), "limits")); + assert!(mentions(&strip("mod limits;", false), "limits")); + assert!(mentions( + &strip("mod limits;\nlet b = Level2Budget::declared();", true), + "Level2Budget" + )); +} + +#[test] +fn every_shipped_module_but_the_budget_itself_is_scanned() { + // A hardcoded file list rots into a list of the files someone + // remembered, so the real guard walks the crate instead: every shipped + // `.rs` under `swang/src` is read, and any that names the budget must be + // one of the three that legitimately may. + let root = concat!(env!("CARGO_MANIFEST_DIR"), "/src"); + let mut scanned = 0_u32; + for path in rust_sources(Path::new(root)) { + let relative = path.strip_prefix(root).unwrap_or(&path); + if is_exempt(relative) { + continue; + } + // Only ever for the failure message — never for the decision. + let shown = relative.display().to_string(); + let code = strip( + &read_to_string(&path).expect("a shipped source"), + relative == Path::new("syntax.rs"), + ); + scanned = scanned.saturating_add(1); + for budget_name in BUDGET_NAMES { + assert!( + !mentions(&code, budget_name), + "{shown} names `{budget_name}`, and it is not one of the \ + modules allowed to: {EXEMPT:?}" + ); + } + } + assert!(scanned > 10, "only {scanned} modules were walked"); +} + +/// The two modules that may name the budget freely: the budget itself and +/// its tests. `syntax.rs` is deliberately absent — it is scanned, with only +/// its `mod limits;` declaration permitted. +/// +/// When SWG-4A-06 adds level-2-specific modules (`parser/v2.rs` and the +/// like), those go on this list explicitly, one at a time. Shared dispatch +/// never joins it: a budget consulted before the level branch is a level-1 +/// bound, whatever file it lives in. +const EXEMPT: &[&[&str]] = &[&["syntax", "limits.rs"], &["syntax", "tests.rs"]]; + +/// Whether a path relative to `swang/src` is one of the exempt modules. +/// +/// Compared component by component. A separator is not the same character +/// on every platform this crate builds for, so the exemption must never be +/// decided by a string that contains one. +fn is_exempt(relative: &Path) -> bool { + EXEMPT.iter().any(|parts| { + relative + .components() + .map(Component::as_os_str) + .eq(parts.iter().copied().map(OsStr::new)) + }) +} + +/// Every `.rs` file under `dir`, recursively, in a deterministic order. +fn rust_sources(dir: &Path) -> Vec { + let mut found = Vec::new(); + let mut entries: Vec = read_dir(dir) + .expect("the crate source directory") + .map(|e| e.expect("a directory entry").path()) + .collect(); + entries.sort(); + for path in entries { + if path.is_dir() { + found.extend(rust_sources(&path)); + } else if path.extension().is_some_and(|e| e == "rs") { + found.push(path); + } + } + found +}