Repository navigation
fix(typescript): parse variance annotations (<in T>, <out T>, <in out T>) - #3226
inth3shadows wants to merge 2 commits into
Conversation
…n out T>`) Vendor tree-sitter-typescript v0.23.2 with a grammar patch, regenerated with tree-sitter-cli 0.25.10, for both the wasm and the kernel (which now compiles the same C instead of the crate). `type_parameter` takes an optional `in` / `out` / `in out`, with `const` on either side; `out` stays usable as a parameter name (upstream colbymchenry#364's rule + brajevicm@9fcaa13). Known limit: a type or JSX tag literally named `out`/`in` right after an expression `<` (`<out>x`, `<out>…</out>`) no longer parses; accepting `out` as a type name everywhere instead broke `[out: T]` tuple labels. Effect: 239 -> 185 files with a parse error, 54 newly clean, none newly broken; Zod 4 -> 0. VS Code src/ and Excalidraw (.ts/.tsx) parse to identical trees. Provenance in docs/grammars/tree-sitter-typescript.md.
…g claims Review follow-ups on the variance patch: assert tree shape (call_expression type arguments, mapped-type and type-predicate names, tuple labels) rather than only hasError; make the extraction test fail without the patch; scope the indexing setup to the test that uses it; drop the next-line-method guard (it belongs with the colbymchenry#335 scanner change). The changelog no longer names Effect files that colbymchenry#335 still blocks, and the docs list the `<out.Provider>` tsx limit, the real over-acceptance (`<out in>`), and that the second `const` is ours rather than colbymchenry#364's.
… README cites fix PRs colbymchenry#361/colbymchenry#364
Mirror colbymchenry#3226: parse TS variance annotations
|
I wrote a CI check for this PR on my fork, plus two small follow-ups on the PR itself. Differential parse check. tsc is treated as ground truth:
The fixtures also record grammar gaps still open on the vendored grammar: Two things on this PR:
|
What
TypeScript 4.7 variance annotations, such as
interface Stream<out A, in E>,<in out T>and<const in T>, are parse errors in tree-sitter-typescript v0.23.2. That is upstream tree-sitter/tree-sitter-typescript#370. Two upstream fix PRs are open, #361 and #364, and both breaktype F<out> = out[]. The repo has had no commit since 2025-01-30, so this carries a patch, following the existingdocs/grammars/*.patchpattern.How
docs/grammars/tree-sitter-typescript.patchapplies to v0.23.2 (f975a62) and changes one rule.type_parametertakes an optionalin/out/in out, withconston either side.outstays a valid parameter name through brajevicm/tree-sitter-typescript@9fcaa13's alias. The external scanner is untouched.tree-sitter-typescriptcrate. It compiles the same generated C fromcodegraph-kernel/grammars/typescript/, which is laid out like upstream soscanner.c's../../common/scanner.hresolves without edits. The wasm is built from the same sources, sokernel-grammar-parityholds.docs/grammars/tree-sitter-typescript.md.Measured
These figures compare the previous wasm with the patched one (web-tree-sitter 0.25.10). "Trees changed" compares node type, field and byte range for every node.
src/(.ts / .tsx)A review pass also ran 1,287 more .ts/.tsx files that use
in/outafter<,,or(. None broke and 24 were fixed.Regenerating the unpatched grammar with 0.25.10 gives the same trees as the previous wasm, so regeneration on its own changes nothing. The upstream corpus passes before and after (112/112).
Limits, stated plainly:
Schema.tsandStream.ts, still parse as one whole-fileERROR. Their next error is a different upstream bug, bug: anonymous generic call signatures fail to parse when separated only by line breaks tree-sitter/tree-sitter-typescript#335: a generic call signature on its own line in an object type. A fix for that comes in a separate PR.in/outnow lex as keywords where an expression<could open type parameters. So a type or JSX tag literally namedout/inin that spot no longer parses:<out>x, and in tsx<out>…</out>,<in>…</in>and<out.Provider>. I tried two ways to keep them. Acceptingoutas a type name everywhere broke[out: T]tuple labels, which Effect uses. Aliasing it in JSX gave GLR conflicts through every attribute. Neither shipped.Tests
__tests__/typescript-variance-parse.test.tschecks tree shape, not justhasError. It checks the modifier tokens and name of each type parameter in typescript and tsx,outas a parameter name, tuple label, mapped-type key, index-signature key, type-predicate subject and type argument, and an indexed file's declarations. 5 of its 7 tests fail on the old grammar, and all pass with the patch.npm run build:kernel),kernel-grammar-parity,kernel-tsjs-parity,kernel-scaffold,grammar-wasm-bytesand the new test give 144/144.tsc --noEmitexits 0.ui-server-api"stays fast" timing test went over its 100 ms budget once under full-suite load in an earlier run and passed 3/3 alone.EXTRACTION_VERSIONandKERNEL_ABI_VERSIONare not bumped, matching the kotlin (#2173) and swift (#2683) grammar patches. Your call if a re-index hint is wanted.Re-index TypeScript projects after upgrading.