Marshal spacing as the output buffer it is

[henrikottesorensen]

Rebased onto #9

The first 15 commits are that PR. Review only the last one, fix: Marshal spacing as the output buffer it is.

Merge #9 first and this reduces to a single commit.

Backlog item 7 from the audit notes: "Fix spacing, or hard-code null and document why." This fixes it, as an output buffer.

What was wrong

liblouis's spacing is in/out — it answers in the same buffer it reads the request from. The wrapper declared it as string? with UTF-8 marshalling, which produced two separate defects:

1. The answer was unreachable. liblouis dutifully computed spacing information and wrote it into the marshaller's temporary buffer, which is freed after the call. A .NET string is immutable, so nothing could ever be copied back. The parameter looked like it worked — no error, translation succeeded — but no caller could ever receive spacing information. It was a feature that could not function.

2. The buffer was undersized, in the direction nobody looked at. The audit notes recorded this as a one-byte overrun on the forward path. Reading the implementation, the back-translation path is much worse.

The header is wrong about the buffer size

liblouis.h.in promises spacing need only be "NULL or at least inlen elements long". The two directions disagree, and neither matches that:

Direction	What it writes	Needs
Forward	memcpy(srcSpacing, destSpacing, input.length) then srcSpacing[input.length] = 0 (lou_translateString.c:1384-1385)	inlen + 1
Backward	memset(spacebuf, '*', *outlen) up front, then writes per output cell (lou_backTranslateString.c:229, 1073)	outlen

So back-translation overran by outlen - inlen bytes. For a 27-character braille input with an outlen of 108 — an ordinary call, and exactly what the round-trip in the new tests does — that is a 80-byte native heap overrun, not one byte.

Buffers are now sized max(inputLength, outputLength) + 1, which is the same shape PrepareTypeFormBuffer already uses for the identical problem in #9.

Two more details the header does not mention: it is a char buffer rather than widechar, and liblouis indexes it in widechars, so a per-char request has to collapse surrogate pairs on the way in. Counting in UTF-16 units would describe a longer buffer than the one allocated — the same trap as the widechar length fixes in #9.

How the answer is reported

TranslatedString.OutputSpacing, following OutputDots78 from #9. That is not an arbitrary choice: the typeform write-back has the same shape — in/out, indexed per output cell — and the same reason it cannot be written back into the caller's input-sized argument. Reusing the pattern keeps all four public signatures unchanged, so this is not a breaking change; string? spacing stays exactly as it was.

The two overloads that return a bare string have nowhere to report it, so they compute and discard it, as they already do for OutputDots78. Their buffers still have to be sized correctly, and are.

Values are one char per output cell: '*' where liblouis reported nothing, an ASCII digit carried over from the input character that produced the cell, or '1' where back-translation inserted a space. Verified:

input    (34): Anden linje, med kursiveret tekst.
request  (34): 0000003330000000000000000000000000
output   (27): @anç linje, m kursi#rò ükz.
spacing  (27): **0003330000000000000000000

The three '3's land on output cells 5–7 — exactly the cells produced by the three input characters that were marked.

One limitation kept rather than papered over

Forward translation copies its answer back over only the first inlen bytes, so when a translation grows the text the cells past the input's length hold no answer, and OutputSpacing is shorter than Output. Back-translation reports the full output. That asymmetry is upstream's, not the wrapper's; it is documented on the property and pinned by a test in each direction rather than hidden behind padding.

A leading 'X' in the request tells liblouis to skip the computation entirely (lou_translateString.c:1203). That now surfaces honestly as OutputSpacing == null instead of echoing the caller's own request back.

Tests

Nine cases in LibLouis.NET.Test/SpacingTests.cs:

• per-output-cell reporting, cross-checked against InputPosition
• null when not requested
• the 'X' disable sentinel
• the back-translation round trip that was the 80-byte overrun
• surrogate-pair collapsing (one entry per character, not per UTF-16 unit)
• the request-length guard
• both non-position overloads, forward and backward

70 tests pass.

Unchanged from 3.33.0 to 3.38.0 — the 3.38 source is identical here apart from an added pos < input->length bounds check, so #11 does not affect this.

Worth reporting upstream

Two things, alongside the table-compilation stack depth already queued: the header's documented spacing size is wrong for back-translation in a way that overruns a conforming caller's buffer, and memcpy(srcSpacing, destSpacing, input.length) reads out of bounds of liblouis's own destSpacing when inlen exceeds the allocation sized from outlen.

🤖 Generated with Claude Code

[henrikottesorensen]

Superseded by #19.

The head branch has moved from the henrikottesorensen/LibLouis.NET fork onto Notalib/LibLouis.NET, because GitHub's stacked pull requests require the head and base branches to live in the same repository (gh-stack#46). No commits or content were changed in the move.