Skip to content

Run the upstream braille specs against the wrapper - #21

Open
henrikottesorensen wants to merge 1 commit into
chore/drop-stale-test-tablesfrom
yaml-specs
Open

Run the upstream braille specs against the wrapper#21
henrikottesorensen wants to merge 1 commit into
chore/drop-stale-test-tablesfrom
yaml-specs

Conversation

@henrikottesorensen

Copy link
Copy Markdown
Collaborator

Stacked on #10. This PR's base is that branch, so the diff and commit list below are only this change. GitHub will retarget it to main automatically when #10 merges.

Chain: #9#10#15#16

Runs liblouis's own test data through the managed wrapper: 10,524 cases, both directions, with expectations written by upstream rather than invented here.

Spec Cases Result
da-dk-6dot.yaml 1,609 pass
da-dk-8dot.yaml 2,100 pass
da-dk_1993.yaml 6,815 pass

5,855 entries expand to 10,524 cases because consecutive table: keys accumulate and a tests: block runs once per accumulated table, in each direction.

Why it needs a reader rather than a deserialiser

These files are not YAML mappings. One document repeats table:, flags: and tests: at the same level — duplicate keys, which a DOM deserialiser rejects or silently collapses. lou_checkyaml treats the file as an event stream where each key mutates parser state (tools/lou_checkyaml.c:1087-1139), so this reader does the same over YamlDotNet's IParser.

Anything it does not model throws rather than being skipped, so a spec using an unsupported construct fails loudly instead of quietly testing less than it appears to.

Two format details that are easy to get wrong

Both of these were wrong first, and the specs caught them:

  • The backward leg of bothDirections swaps input and expected, because back-translation should turn the braille back into the text. An explicit testmode: backward does not swap — the entry is already written braille-first. lou_checkyaml.c:892 against :900. Treating them alike asked the wrapper to back-translate Danish text into Danish text, and failed ~2,800 cases.
  • A display table can be an inline table, written as a YAML block scalar, to include the standard one and override a character. That arrives as an ordinary scalar event, so it was read as a file name, and every test under it failed with a path of tables/include unicode-without-blank.dis. They are now written out beside the tables, where their own include lines resolve. This accounted for all 140 remaining mismatches, from just two spec lines: TAB and LF.

What it took to unblock

The harness was written earlier and shipped Skipped, because resolving the specs' table queries needs lou_findTable, whose returned string the marshaller freed with the wrong allocator — the fifth consecutive call hung the process. That is audit item 3, fixed in #9, and the evidence for it came from this harness. With #9 the whole suite runs in 375 ms.

It also needs #10: the specs must be checked against upstream tables, and #10 is what stops Nota's forks shadowing them in tables/. Before that, 2,699 forward cases failed against tables they were never written for.

Coverage this adds

bothDirections exercises BackTranslate, and the 19 table queries exercise FindTable and IndexTables — three of the six public methods that had no coverage at all.

66 tests pass in total on the stack.

Why only Danish, and what it would take to widen it

Upstream ships 150 specs. Eight are dictionary harnesses of 200k+ cases each, which want to be opt-in rather than part of every run. That leaves 142 specs, ~98,600 entries, across roughly 80 languages.

I ran the reader over all 142. 122 parse fine already. The 20 that do not break down as:

Blocker Specs
[description, input, expected] entries, where the first element is a label rather than the input 10
per-test mode 4
per-test outputPos 2
per-test inputPos 1
testmode: display 1
testmode: hyphenate 1
a multi-line double-quoted scalar YamlDotNet rejects 1

So widening is mostly a matter of supporting one more entry shape, plus deciding whether to implement or account for a handful of options. hyphenate and display map to Hyphenate and DotsToCharacters/CharactersToDots, and inputPos/outputPos to the position arrays — all wrapper surface with no coverage today, so implementing rather than skipping them is worth considering.

That is being done as a follow-up rather than here, so this PR stays reviewable.


Recreated from #15 to move the head branch onto Notalib/LibLouis.NET, which GitHub stacked PRs require (stacks cannot span forks).

Adds a reader for liblouis's braille spec files and runs the three Danish
ones through the managed wrapper. 10524 cases, in both directions, whose
expectations were written by upstream rather than invented here.

The files are not YAML mappings and cannot be deserialised: one document
repeats table, flags and tests at the same level, and lou_checkyaml
treats the file as an event stream where each key mutates parser state
(tools/lou_checkyaml.c:1087-1139). The reader does the same over
YamlDotNet's IParser. Consecutive table keys accumulate rather than
replace, so a tests block runs once per accumulated table, which is why
5855 entries expand to 10524 cases.

Two details of the format are easy to get wrong and both were, before the
specs caught them:

- the backward leg of bothDirections swaps input and expected, because
  back translation should turn the braille back into the text, while an
  explicit testmode: backward does not (lou_checkyaml.c:892 against 900).
- a display table can be an inline table written as a block scalar rather
  than a file name, to include the standard one and override a character.
  That arrives as an ordinary scalar, so it was read as a path, and every
  test under it failed to translate. Those are now written out beside the
  tables, where their own includes resolve.

Anything the reader does not model throws rather than being skipped, so a
spec using an unsupported construct fails loudly instead of quietly
testing less than it appears to.

One test per spec file rather than per case: ten thousand xunit cases
makes discovery slow and buries a real regression, where a single failure
listing every mismatch does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants