Honor TokenReader subrange bounds in peek and backtrack.

[Christopher Pruijsen (cpruijsen)]

TokenReader.peekToken(), peekPreviousTokenKind(), and backtrackToMarker() now honor the same [_readerStartIndex, _readerEndIndex) window that peekTokenKind() / peekTokenAfterKind() / peekTokenAfterAfterKind() already enforce. Fixes #481: an embedded reader could observe the token before its start or after its end, and backtrackToMarker could rewind into the outer stream. TokenReader is not a public export. Current NodeParser hot paths already call peekTokenKind() first, which is why existing parser snapshots did not fail.

Decision

peekToken() past the window returns the stream's EndOfInput token (tokens[tokens.length - 1]); peekPreviousTokenKind() treats _readerStartIndex as start-of-input; backtrackToMarker throws if marker < _readerStartIndex.

Alternative: throw from peekToken() (like readToken()), or clamp an out-of-range marker to the window start. Open PR #482 takes the throw-from-peekToken() option.

readToken() already states that peek must always return a valid Token, and the other peek methods already signal EndOfInput at the window edge. Expired markers already throw as a parser bug. Can switch to throw-on-peek or clamp-on-backtrack.

Test plan

• New tsdoc/src/parser/__tests__/TokenReader.test.ts fails without the source change and passes with it (embedded predecessor leak, next-token leak, empty TokenSequence.createEmpty, out-of-range backtrack).
• Full @microsoft/tsdoc Heft test run (typecheck / ESLint / API Extractor / 259 Jest tests).
• rush change --verify on CI (change file included).
• Optional: playground / api-demo smoke; no parser grammar change.

Change log

Rush change file: common/changes/@microsoft/tsdoc/fix-tokenreader-bounds_2026-09-11-13-00.json (patch).