Fix heap corruption on instructions that outgrow the parse tree

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Context.translate corrupts the heap on ordinary encodings. ParserContext keeps an instruction's parse tree in three fixed arrays and allocateOperand indexes all three without checking any of them, so PowerPC AltiVec operands and ARM and NDS32 register lists write past the end. Translation still returns correct p-code, and the process dies later in an unrelated malloc or free.

A node's operand array is now sized to the constructor attached to it, the node array and breadcrumb trail hold the largest tree the shipped languages build, and allocateOperand raises BadDataError instead of overrunning them, which is already how oneInstruction reports undecodable input.

The regression decodes the PowerPC, ARM and NDS32 instructions in child interpreters, since master crashes rather than fails. These sources are vendored from Ghidra unchanged, so the defect is upstream too.

Validation: #289 (comment)

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 54d3ad52d932d7a7cb4e7287778d47a0cdd860a4 against baseline 559aacdc9d363fd19477d9daa40721279cd99248.

• Regression: this branch's tests/test_pypcode.py against a baseline build — Ran 45 tests ... FAILED (failures=8), every failure in ParseTreeTests and the other 41 tests passing; on head pytest tests/test_pypcode.py -k ParseTree is 4 passed and 6 subtests passed
• Full suite: pytest — 50 passed, 193 subtests passed, 0 skipped
• Lint/type: pre-commit run --all-files — black, ruff, mypy, pylint and clang-format all pass and no hook rewrote a file. The repository config excludes pypcode/sleigh, so the vendored sources keep upstream formatting
• Workspace gate: pypcode built and tested from this branch; archinfo, pyvex, cle, claripy, angr and angr-management are wheel-installed and their suites skipped
• Build workflow jobs run locally: python -m build --sdist, the wheel job's python -m unittest discover -s tests, and make html coverage

Reproducers, one instruction per process, Context.translate then Context.disassemble, five runs each and identical every run. Register lists are abbreviated in the last column.

Language	Bytes	Baseline	Head
PowerPC:BE:32:default	10 00 00 00	SIGABRT, double free or corruption (!prev)	vaddubm v0,v0,v0
PowerPC:BE:32:default	10 00 00 40	SIGSEGV	vadduhm v0,v0,v0
PowerPC:BE:32:default	10 00 04 40	SIGSEGV	vsubuhm v0,v0,v0
NDS32:LE:32:default	3b c5 37 45	SIGABRT, free(): invalid size	lmwa.bim fp, [s4], s7, 0xd
ARM:LE:32:v7	14 0a 90 ec	SIGABRT, free(): invalid size	vldmia r0,{s0,s1,...,s19}
ARM:LE:32:v7	1f 0a 90 ec	SIGSEGV	vldmia r0,{s0,s1,...,s30}
JVM:BE:32:default	ab	SIGSEGV	BadDataError: Instruction parse tree is too large

Which glibc check fires, and whether the process aborts or faults, depends on heap layout and varies between builds; the PowerPC and ARM deaths happen after translate has already returned correct p-code.

Tree sizes, from a build carrying a counter in allocateOperand, against the allotment of 75 nodes and 32 breadcrumb entries the baseline hands out:

Instruction	Nodes	Levels
vaddubm	60	3
vadduhm, vsubuhm	34	3
vldmia r0,{s0-s19}	76	25
vldmia r0,{s0-s30}	109	36
lmwa.bim fp,[s4],s7,0xd	76	22

The operand counts come from the shipped specs: altivec.sinc gives vaddubm_part1 and vaddubm_part2 24 operands each and vadduhm and vsubuhm 27, against an allotment of 20.

• Limits: 20,000 random 16-byte inputs against each of the 187 shipped languages, one process per language, on a build with the same counter — the largest tree outside JVM is 120 nodes at 36 levels (NDS32:BE:32:default), and JVM:BE:32:default is the only language that ever reaches a limit, refusing 157 of its 20,000 inputs
• Decoding unchanged: 400 random 16-byte inputs per language, one process per language, hashing mnemonic, operand text, instruction length and every p-code op — all 181 languages that survive both builds hash identically and none differ. The LOAD/STORE space operand holds an AddrSpace pointer, so it is resolved to a space name before hashing; without that the hash moves with ASLR on both builds alike
• Compiler unaffected: the 149 .sla files the build regenerates are byte-identical to baseline
• Throughput: x86:LE:64:default, 400 KiB of random bytes decoded at every byte offset, 358,037 successful decodes on both builds, best of 7 — 0.516 s baseline, 0.517 s head
• Memory: 20 Context objects per language, resident set divided by 20 — x86:LE:64:default 26,008 to 26,247 KiB, MIPS:BE:32:default 9,001 to 9,240 KiB, sparc:BE:32:default 1,788 to 2,023 KiB, so roughly 235 KiB more per Context
• Error path: translate and disassemble already suppress a BadDataError raised after the first instruction, so a refused instruction ends the block and the instructions before it are kept. Refusing abandons a half-built tree in a cached ParserContext, and test_context_is_reusable_after_a_refusal covers that the Context keeps working: neighbouring addresses in the same cache slot still decode and asking again at the refused address raises
• angr: CFGFast over blobs of the PowerPC, NDS32 and JVM reproducers with angr 9.3.3.dev0 — baseline dies on all three, head completes all three. angr already maps pypcode.BadDataError to Ijk_NoDecode. The ARM reproducer cannot be checked this way: CFGFast on an ArchPcode ARM blob raises AttributeError: 'CFGArchOptions' object has no attribute 'switch_mode_on_nodecode' on both builds, which is an unrelated angr defect

Caveats:

• JVM lookupswitch still does not decode. It raises now instead of writing past the node array, which is containment rather than support
• Both bounds are load-bearing. At the shipped node cap the JVM recursion hits the node bound first; rebuilt with only the node cap raised to 100,000, the same input reports Instruction parse tree is too deep instead, so the breadcrumb trail needs its own check
• The sparc:BE:32:default and sparc:BE:64:default baseline deaths in the sweep above are a different defect — those trees never exceed 11 nodes, and it is the delay-slot UnimplError use-after-free that Fix segfault when a delay-slot instruction has no p-code #288 fixes. This change grows ParserWalker, which moves the stack enough that Fix segfault when a delay-slot instruction has no p-code #288's reproducers stop faulting on head; it does not repair them, and Fix segfault when a delay-slot instruction has no p-code #288 is still the fix
• The counter-build measurements and the .sla and angr checks were taken at 478e6635c8b1b11f7a5b182268cf677bc6cb1532; the only change since then in built code is comment text, and everything else above was rerun at the head named here

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.82%. Comparing base (559aacd) to head (54d3ad5).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #289   +/-   ##
=======================================
  Coverage   86.82%   86.82%           
=======================================
  Files           5        5           
  Lines         516      516           
  Branches       82       82           
=======================================
  Hits          448      448           
  Misses         26       26           
  Partials       42       42           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.