Skip to content

Widen the braille specs from Danish to 102 upstream specs - #16

Closed
henrikottesorensen wants to merge 3 commits into
Notalib:yaml-specsfrom
henrikottesorensen:yaml-specs-all
Closed

Widen the braille specs from Danish to 102 upstream specs#16
henrikottesorensen wants to merge 3 commits into
Notalib:yaml-specsfrom
henrikottesorensen:yaml-specs-all

Conversation

@henrikottesorensen

@henrikottesorensen henrikottesorensen commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #15. This PR's base is that branch, so the diff and commit list below are only this change. GitHub will retarget it to main automatically when #15 merges.

Chain: #9#10#15#16

Runs liblouis's own expectations for roughly 60 languages through the wrapper instead of only Danish. 102 specs, all passing.

Five things the reader did not model

Every one was found by specs failing, not by reading the format:

  • a test entry can lead with a description — [label, input, expected]
  • a translation table can be written inline as a block scalar rather than named. nemeth.yaml failed 133 of 133 on this alone
  • so can a display table, same cause
  • a table can be named by file rather than by query, in which case it must not go through lou_findTable
  • typeform, mode, inputPos, outputPos and cursorPos change what the expected output means, so those cases are counted and dropped rather than run against the wrong expectation

That last one is the design point. BrailleSpec.SkippedConstructs records what was recognised but not driven, so coverage that is not happening stays visible instead of quietly disappearing:

Construct Occurrences Maps to
inputPos 5,071 position arrays
outputPos 5,070 position arrays
typeform 522 TypeForm
cursorPos 19 cursor handling
mode 19 TranslationMode

Anything outside that list still throws. Driving these would widen the corpus and cover wrapper surface that has no tests today — the same job twice over.

Why each spec runs on its own large-stack thread

Compiling a table can recurse deeply. ancient-languages-borger.utb needs between 640 KB and 768 KB, which is more than the test host gives a test, and a stack overflow kills the process rather than failing a test.

It is compilation, not translation — once a table list is compiled, translating through it runs in 128 KB. Nothing about the input matters; ASCII overflows the same as non-BMP. And liblouis caches compiled tables process-wide, keyed by table-list string and never evicted, so without a large stack somewhere the outcome depends on which test happened to compile a given table first.

Worth knowing beyond the tests: the same failure is reachable in production. An ASP.NET request thread has about 1 MB. A service that compiles a table list for the first time on a request thread can overflow the same way, uncatchably. Compiling table lists at startup avoids it. Diagnosed jointly with the P/Invoke audit session, after we both drew wrong conclusions from experiments that were really measuring the compile cache.

40 specs held back

Not because they are wrong — because nobody has established yet whether the disagreement is the harness or the wrapper, and a suite that is expected to be red is worse than a smaller green one. Listed with reasons in braille-specs/README.md:

  • 29 fail on table resolution, and are probably one root cause rather than 29: 22 report no table matching a query, 7 resolve to a table other than the one __assert-match names. The queries look well formed (language:bn grade:1), so the likely cause is which tables reach lou_indexTables or which liblouis manages to analyse. Cheapest place to start.
  • 10 disagree on translation output, a fraction of cases each — no.yaml 167/868, ru.yaml 39/140. The interesting group: either an unmodelled per-case option or a real difference.
  • 1 does not parse: a multi-line double-quoted scalar YamlDotNet rejects.

🤖 Generated with Claude Code

henrikosorensen and others added 3 commits August 5, 2026 12:31
LibLouis.NET.Test/tables held thirty of Nota's Danish tables while
LibLouis.NET.Tables copies the upstream set into the same output
directory. Twenty two of the thirty share a file name with an upstream
table and differ from it, so which copy a test got depended on MSBuild
item ordering, and nothing said so anywhere.

Moves them to nota-tables/, kept separate in the output, and points every
test at that path. Each test now states which set it means, and the
upstream tables are no longer shadowed, which is what lets upstream's
braille specs be checked against them.

Nota's tables include seven general upstream tables, and liblouis
resolves an include relative to the directory of the table doing the
including, so those are copied in alongside. They come from the staged
upstream set rather than being committed, so they cannot drift from it.
NotaTablesAreSelfContained asserts that list stays complete, since an
explicit list goes stale the moment a table gains an include.

The second guard checks that none of Nota's tables reach tables/. Sharing
a file name across the two directories is expected - most of Nota's are
forks of an upstream table of the same name - so the tables that exist
nowhere upstream are the canary: if one appears in tables/, the two sets
are being copied to the same place again.

No table content changes. Twenty three of the thirty are reachable from
no test - they are a stale copy of the set the application ships - but
they are kept and documented rather than deleted, because deciding what
is canonical is not this change's business.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds a reader for liblouis's braille spec files and runs the three Danish
ones through the managed wrapper. 10524 cases, in both directions, whose
expectations were written by upstream rather than invented here.

The files are not YAML mappings and cannot be deserialised: one document
repeats table, flags and tests at the same level, and lou_checkyaml
treats the file as an event stream where each key mutates parser state
(tools/lou_checkyaml.c:1087-1139). The reader does the same over
YamlDotNet's IParser. Consecutive table keys accumulate rather than
replace, so a tests block runs once per accumulated table, which is why
5855 entries expand to 10524 cases.

Two details of the format are easy to get wrong and both were, before the
specs caught them:

- the backward leg of bothDirections swaps input and expected, because
  back translation should turn the braille back into the text, while an
  explicit testmode: backward does not (lou_checkyaml.c:892 against 900).
- a display table can be an inline table written as a block scalar rather
  than a file name, to include the standard one and override a character.
  That arrives as an ordinary scalar, so it was read as a path, and every
  test under it failed to translate. Those are now written out beside the
  tables, where their own includes resolve.

Anything the reader does not model throws rather than being skipped, so a
spec using an unsupported construct fails loudly instead of quietly
testing less than it appears to.

One test per spec file rather than per case: ten thousand xunit cases
makes discovery slow and buries a real regression, where a single failure
listing every mismatch does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Runs liblouis's own expectations for roughly 60 languages through the
wrapper instead of only Danish. 102 specs, all passing.

Five things the reader did not model, each found by specs failing rather
than by reading the format:

- a test entry can lead with a description, [label, input, expected]
- a translation table can be written inline as a block scalar, not just
  named. nemeth.yaml failed 133 of 133 on this alone
- so can a display table, for the same reason
- a table can be named by file rather than by query, in which case it
  must not go through lou_findTable
- typeform, mode, inputPos, outputPos and cursorPos change what the
  expected output means, so those cases are counted and dropped rather
  than run against the wrong expectation

The counting is the point: BrailleSpec.SkippedConstructs records what was
recognised but not driven, so coverage that is not happening stays
visible. Anything outside that list still throws.

Each spec runs on a thread with a large stack. Compiling a table can
recurse deeply - ancient-languages-borger.utb needs between 640KB and
768KB, more than the test host gives a test - and a stack overflow kills
the process rather than failing a test. It is compilation, not
translation: once a table list is compiled, translating through it runs
in 128KB. Nothing about the input matters, and liblouis caches compiled
tables process wide, so without a large stack somewhere the outcome
depends on which test compiled a table first.

40 specs are held back with their reasons written down in the README. 29
of them fail on table resolution and are probably one root cause rather
than 29.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
henrikottesorensen pushed a commit to henrikottesorensen/LibLouis.NET that referenced this pull request Aug 7, 2026
Ten pull requests now, Notalib#16 among them rather than pending. The forty
held-back specs are characterised: twenty nine are table resolution and
probably one root cause, ten are genuine output disagreements, one does
not parse.

Merging the queue moves to the top of the backlog. Nothing has merged
since Notalib#8, and Notalib#16 now sits four deep behind Notalib#9.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@henrikottesorensen
henrikottesorensen changed the base branch from main to yaml-specs August 12, 2026 06:27
@henrikottesorensen
henrikottesorensen requested a review from a team August 12, 2026 06:35
@henrikottesorensen

Copy link
Copy Markdown
Collaborator Author

Superseded by #22.

The head branch has moved from the henrikottesorensen/LibLouis.NET fork onto Notalib/LibLouis.NET, because GitHub's stacked pull requests require the head and base branches to live in the same repository (gh-stack#46). No commits or content were changed in the move.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants