Skip to content

Integrate modernization prep and fix save, bank and texture decoding - #39

Merged
BuffJesus merged 25 commits into
mainfrom
integration/modern-fork-prep-20261024
Oct 24, 2026
Merged

BuffJesus merged 25 commits into
mainfrom
integration/modern-fork-prep-20261024

Conversation

@BuffJesus

@BuffJesus BuffJesus commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Bring the public modernization groundwork from #36 onto current main, retaining
its history and the CI repairs already on main. This includes the save, bank,
codec, compatibility, platform and texture modules, their tools/tests, and
modernization documentation. This supersedes #36; its occupied branch is preserved.

Fix issues found during integration and review:

  • Python save text: emit correct empty/short strings and reject truncated framing,
    payloads and tails instead of returning incomplete text and invalid offsets.
  • LZO/zlib decoding: enforce declared output bounds while decoding and avoid
    allocating output from the declared size alone. Empty zlib chunks now decode,
    and trailing bytes inside a compressed chunk are rejected.
  • Bank decoding: reject impossible entry counts before reserving storage and
    handle a null input as truncated data.
  • DXT decoding: include partial right/bottom blocks and reject truncated block
    data before allocating the output image.

Synthetic regressions reproduce the pre-fix failures and cover signed-length
boundaries, adjacent fields, empty chunks, output bounds, minimum bank entries
and DXT1/DXT3 edge blocks. Also remove one trailing documentation blank line.

Local validation: GCC 15.2 Release with LZO OFF and ON, and MSVC 19.44 x64 Release
with LZO OFF, each pass full builds and 10/10 CTest tests. Python source-only
selection passes 178 tests and 122 subtests (33 conditional installed-game tests
explicitly skipped; 10 exclusions from the existing vc71/retail name filter).
README consistency, documentation links/findings/root and whitespace checks pass.
All three hosted checks pass for review head 632c28b: Python/docs consistency,
Linux GCC/LZO ON, and Windows MSVC x64. A bounded independent review of the
eight-file C++ correction found no merge blocker. No game/corpus validation
is claimed. No private owner source, evidence or build outputs are included.

BuffJesus and others added 25 commits October 17, 2026 14:58
rebuild/modern/save reads and rewrites retail saves and Profile.bin with portable C++23 types;
every game-written save re-encodes byte for byte. LZO for the quest/faction text is an optional
GPL download (FABLE_SAVE_WITH_LZO), never vendored. rebuild/modern/bank round-trips .big bank
archives with payload views and the BIG.md minimal-repack writer. modern-tree.yml builds and
tests the tree on Ubuntu GCC (with LZO) and Windows MSVC x64 (without).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ffer patcher

modern_compile_census.py syntax-checks every reconstructed TU with VS 2022 as x86 and x64 and
reports readiness by fix and layout drift; platform_surface, persisted_types and the D3D9
census tools inventory what the fork must replace; mouse_buffer_patch.py writes a copy of the
Steam exe with a larger DirectInput mouse buffer (2 bytes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retail reads the blob length as a signed 16-bit value: -1 means a 32-bit length follows and
zero or negative means the text is stored raw. The parser now reads both forms, and the
writer no longer emits 16-bit lengths above 0x7FFF that the game would misread as stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds docs/modern (plan, compile census, layout drift, persisted types, D3D9 and platform
surfaces, frame timing) and the 2026-10-16/17 modernization journals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…doc not yet on main

The world ticks at exactly 15.000 Hz with four renders per world frame at 60 Hz; injected
mouse bursts overflow retail's 256-record DirectInput buffer and drain cleanly at 8,192.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tree

rebuild/modern/codec holds the tree's own LZO1X decoder (moved out of save/). rebuild/modern/texture
decodes the 34-byte texture descriptor, the LZO1X mip 0 and A8R8G8B8/DXT1/DXT3 to RGBA (394/394
frontend.big and 6,319/6,324 textures.big entries; the rest use two unconfirmed formats).
fable_title_view composes the two retail title sprites and presents them in a 64-bit D3D11
window, with --capture reading the presented frame back to a PNG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mpression policy

The fork's codec gains the clean-room LZO1X encoder from FableForge, so files the original game
must read no longer need GPL code to be compressed; upstream LZO stays an optional byte-exact
test. New fork formats use zstd/LZ4/zlib.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tructed raster with VS 2022

fable_title_view now has the fork's two planned backends (--api d3d12|vulkan) instead of D3D11;
their captured frames are pixel-identical. modern_title_raster.py compiles the parity lane's 17
reconstructed renderer sources and harness, unmodified, with VS 2022 x86: all 8 frames match
the VC7.1 build byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ical to VC7.1)

The parity lane's 17 reconstructed renderer sources build unmodified with VS 2022 x64 and draw the
retail title byte-identically through 64-bit D3D9; only three 32-bit layout shortcuts in the test
harness are patched, in generated copies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n tests

326 behaviour tests pass under VC7.1; 308 still pass when built with VS 2022 x86 and 179 with
VS 2022 x64. The report groups the rest by cause (bitset layout, asm bakes, 32-bit layouts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n the behaviour census

fable::compat::retail_bitset keeps VC7.1's 32-bit-word storage. With CThing::UsingComponents on it,
315 of the 326 VC7.1-passing reconstructed-function tests pass under VS 2022 x86 (was 308).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ayouts

96 of the 104 x64 runtime failures come from fixtures with 32-bit storage arrays or byte offsets,
not from the reconstructed code under test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… traces to a 32-bit test harness

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MouseAccumulator folds every OS mouse report into the next world tick's frame, so no polling rate
can overflow it; RawMouseWin32 feeds it from WM_INPUT. The injected bursts that overflowed
retail's 256-record DirectInput buffer are all counted (busiest tick 459 reports).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pdate

A world frame runs when the predicted present is more than one 15 Hz period past the current
frame; tests replay the live sample (4 renders per world frame at 60 Hz) and keep the world at
15 Hz at any refresh rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…DX's snapshot plus events

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…separate triggers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@BuffJesus BuffJesus changed the title Integrate modernization prep and repair save text framing Integrate modernization prep and fix save, bank and texture decoding Oct 24, 2026
@BuffJesus
BuffJesus marked this pull request as ready for review October 24, 2026 20:27
@BuffJesus
BuffJesus merged commit f11cfa5 into main Oct 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant