Skip to content

Run the upstream braille specs against the wrapper - #15

Closed
henrikottesorensen wants to merge 2 commits into
Notalib:chore/drop-stale-test-tablesfrom
henrikottesorensen:yaml-specs
Closed

Run the upstream braille specs against the wrapper#15
henrikottesorensen wants to merge 2 commits into
Notalib:chore/drop-stale-test-tablesfrom
henrikottesorensen:yaml-specs

Conversation

@henrikottesorensen

@henrikottesorensen henrikottesorensen commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #10. This PR's base is that branch, so the diff and commit list below are only this change. GitHub will retarget it to main automatically when #10 merges.

Chain: #9#10#15#16

Runs liblouis's own test data through the managed wrapper: 10,524 cases, both directions, with expectations written by upstream rather than invented here.

Spec Cases Result
da-dk-6dot.yaml 1,609 pass
da-dk-8dot.yaml 2,100 pass
da-dk_1993.yaml 6,815 pass

5,855 entries expand to 10,524 cases because consecutive table: keys accumulate and a tests: block runs once per accumulated table, in each direction.

Why it needs a reader rather than a deserialiser

These files are not YAML mappings. One document repeats table:, flags: and tests: at the same level — duplicate keys, which a DOM deserialiser rejects or silently collapses. lou_checkyaml treats the file as an event stream where each key mutates parser state (tools/lou_checkyaml.c:1087-1139), so this reader does the same over YamlDotNet's IParser.

Anything it does not model throws rather than being skipped, so a spec using an unsupported construct fails loudly instead of quietly testing less than it appears to.

Two format details that are easy to get wrong

Both of these were wrong first, and the specs caught them:

  • The backward leg of bothDirections swaps input and expected, because back-translation should turn the braille back into the text. An explicit testmode: backward does not swap — the entry is already written braille-first. lou_checkyaml.c:892 against :900. Treating them alike asked the wrapper to back-translate Danish text into Danish text, and failed ~2,800 cases.
  • A display table can be an inline table, written as a YAML block scalar, to include the standard one and override a character. That arrives as an ordinary scalar event, so it was read as a file name, and every test under it failed with a path of tables/include unicode-without-blank.dis. They are now written out beside the tables, where their own include lines resolve. This accounted for all 140 remaining mismatches, from just two spec lines: TAB and LF.

What it took to unblock

The harness was written earlier and shipped Skipped, because resolving the specs' table queries needs lou_findTable, whose returned string the marshaller freed with the wrong allocator — the fifth consecutive call hung the process. That is audit item 3, fixed in #9, and the evidence for it came from this harness. With #9 the whole suite runs in 375 ms.

It also needs #10: the specs must be checked against upstream tables, and #10 is what stops Nota's forks shadowing them in tables/. Before that, 2,699 forward cases failed against tables they were never written for.

Coverage this adds

bothDirections exercises BackTranslate, and the 19 table queries exercise FindTable and IndexTables — three of the six public methods that had no coverage at all.

66 tests pass in total on the stack.

Why only Danish, and what it would take to widen it

Upstream ships 150 specs. Eight are dictionary harnesses of 200k+ cases each, which want to be opt-in rather than part of every run. That leaves 142 specs, ~98,600 entries, across roughly 80 languages.

I ran the reader over all 142. 122 parse fine already. The 20 that do not break down as:

Blocker Specs
[description, input, expected] entries, where the first element is a label rather than the input 10
per-test mode 4
per-test outputPos 2
per-test inputPos 1
testmode: display 1
testmode: hyphenate 1
a multi-line double-quoted scalar YamlDotNet rejects 1

So widening is mostly a matter of supporting one more entry shape, plus deciding whether to implement or account for a handful of options. hyphenate and display map to Hyphenate and DotsToCharacters/CharactersToDots, and inputPos/outputPos to the position arrays — all wrapper surface with no coverage today, so implementing rather than skipping them is worth considering.

That is being done as a follow-up rather than here, so this PR stays reviewable.

henrikosorensen and others added 2 commits August 5, 2026 12:31
LibLouis.NET.Test/tables held thirty of Nota's Danish tables while
LibLouis.NET.Tables copies the upstream set into the same output
directory. Twenty two of the thirty share a file name with an upstream
table and differ from it, so which copy a test got depended on MSBuild
item ordering, and nothing said so anywhere.

Moves them to nota-tables/, kept separate in the output, and points every
test at that path. Each test now states which set it means, and the
upstream tables are no longer shadowed, which is what lets upstream's
braille specs be checked against them.

Nota's tables include seven general upstream tables, and liblouis
resolves an include relative to the directory of the table doing the
including, so those are copied in alongside. They come from the staged
upstream set rather than being committed, so they cannot drift from it.
NotaTablesAreSelfContained asserts that list stays complete, since an
explicit list goes stale the moment a table gains an include.

The second guard checks that none of Nota's tables reach tables/. Sharing
a file name across the two directories is expected - most of Nota's are
forks of an upstream table of the same name - so the tables that exist
nowhere upstream are the canary: if one appears in tables/, the two sets
are being copied to the same place again.

No table content changes. Twenty three of the thirty are reachable from
no test - they are a stale copy of the set the application ships - but
they are kept and documented rather than deleted, because deciding what
is canonical is not this change's business.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds a reader for liblouis's braille spec files and runs the three Danish
ones through the managed wrapper. 10524 cases, in both directions, whose
expectations were written by upstream rather than invented here.

The files are not YAML mappings and cannot be deserialised: one document
repeats table, flags and tests at the same level, and lou_checkyaml
treats the file as an event stream where each key mutates parser state
(tools/lou_checkyaml.c:1087-1139). The reader does the same over
YamlDotNet's IParser. Consecutive table keys accumulate rather than
replace, so a tests block runs once per accumulated table, which is why
5855 entries expand to 10524 cases.

Two details of the format are easy to get wrong and both were, before the
specs caught them:

- the backward leg of bothDirections swaps input and expected, because
  back translation should turn the braille back into the text, while an
  explicit testmode: backward does not (lou_checkyaml.c:892 against 900).
- a display table can be an inline table written as a block scalar rather
  than a file name, to include the standard one and override a character.
  That arrives as an ordinary scalar, so it was read as a path, and every
  test under it failed to translate. Those are now written out beside the
  tables, where their own includes resolve.

Anything the reader does not model throws rather than being skipped, so a
spec using an unsupported construct fails loudly instead of quietly
testing less than it appears to.

One test per spec file rather than per case: ten thousand xunit cases
makes discovery slow and buries a real regression, where a single failure
listing every mismatch does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@henrikottesorensen

Copy link
Copy Markdown
Collaborator Author

Superseded by #21.

The head branch has moved from the henrikottesorensen/LibLouis.NET fork onto Notalib/LibLouis.NET, because GitHub's stacked pull requests require the head and base branches to live in the same repository (gh-stack#46). No commits or content were changed in the move.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants