Skip to content

Modernization fork prep: save and bank codecs, compile census, frame-timing audit - #36

Merged
BuffJesus merged 21 commits into
mainfrom
modern/fork-prep-2026-10-17
Oct 24, 2026
Merged

BuffJesus merged 21 commits into
mainfrom
modern/fork-prep-2026-10-17

Conversation

@BuffJesus

Copy link
Copy Markdown
Owner

Groundwork for the x64 / modern-compiler fork (docs/modern/MODERNIZATION.md). Nothing here touches the VC7.1 parity tree.

Code (rebuild/modern/, C++23, MSVC x86/x64 and GCC 15 warning-free)

  • save/: retail saves and Profile.bin, every body section typed. All 402 game-written saves and 153 profiles in the local corpus re-encode byte for byte. LZO for the quest/faction text is an optional GPL download (-DFABLE_SAVE_WITH_LZO=ON), never committed; without it the text is written in retail's own stored form.
  • bank/: .big archives with payload views and the BIG.md minimal-repack writer. All 17 bank files in the install round-trip byte for byte; resize and revert edits hold.
  • CI: .github/workflows/modern-tree.yml (Ubuntu GCC with LZO, Windows MSVC x64 without).

Tools (tools/modern/)

  • modern_compile_census.py: VS 2022 syntax check of every TU as x86 and x64, readiness by fix, layout drift for both architectures.
  • mouse_buffer_patch.py: writes a copy of the Steam exe with a larger DirectInput mouse buffer (2 bytes); not yet tested in-game.
  • Platform, persisted-type and D3D9 inventories from the 2026-10-16 prep.

Fix: tools/parse_save_entities.py now reads and writes LZ blob lengths as retail does (signed 16-bit; -1 then u32).

Docs: docs/modern/ (plan, census, frame-timing audit) and the 2026-10-16/17 journals. The matching notes in shared docs (ROADMAP, HANDOFF, GOTCHAS, SAVE.md, BIG.md, reported bugs) stay in the working tree with other sessions' edits and will follow separately.

🤖 Generated with Claude Code

BuffJesus and others added 5 commits October 17, 2026 14:58
rebuild/modern/save reads and rewrites retail saves and Profile.bin with portable C++23 types;
every game-written save re-encodes byte for byte. LZO for the quest/faction text is an optional
GPL download (FABLE_SAVE_WITH_LZO), never vendored. rebuild/modern/bank round-trips .big bank
archives with payload views and the BIG.md minimal-repack writer. modern-tree.yml builds and
tests the tree on Ubuntu GCC (with LZO) and Windows MSVC x64 (without).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ffer patcher

modern_compile_census.py syntax-checks every reconstructed TU with VS 2022 as x86 and x64 and
reports readiness by fix and layout drift; platform_surface, persisted_types and the D3D9
census tools inventory what the fork must replace; mouse_buffer_patch.py writes a copy of the
Steam exe with a larger DirectInput mouse buffer (2 bytes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retail reads the blob length as a signed 16-bit value: -1 means a 32-bit length follows and
zero or negative means the text is stored raw. The parser now reads both forms, and the
writer no longer emits 16-bit lengths above 0x7FFF that the game would misread as stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds docs/modern (plan, compile census, layout drift, persisted types, D3D9 and platform
surfaces, frame timing) and the 2026-10-16/17 modernization journals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…doc not yet on main

The world ticks at exactly 15.000 Hz with four renders per world frame at 60 Hz; injected
mouse bursts overflow retail's 256-record DirectInput buffer and drain cleanly at 8,192.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@BuffJesus

Copy link
Copy Markdown
Owner Author

CI note: modern-tree passes (Ubuntu GCC with LZO, Windows MSVC x64). In docs-consistency, the link check now passes; the remaining failure is the pre-existing 'Pure-Python pipeline tests' step (ModuleNotFoundError: numpy while collecting frontend-render tests), which has failed on main since 2026-09-08 and is untouched by this PR.

BuffJesus and others added 16 commits October 17, 2026 15:43
… fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tree

rebuild/modern/codec holds the tree's own LZO1X decoder (moved out of save/). rebuild/modern/texture
decodes the 34-byte texture descriptor, the LZO1X mip 0 and A8R8G8B8/DXT1/DXT3 to RGBA (394/394
frontend.big and 6,319/6,324 textures.big entries; the rest use two unconfirmed formats).
fable_title_view composes the two retail title sprites and presents them in a 64-bit D3D11
window, with --capture reading the presented frame back to a PNG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mpression policy

The fork's codec gains the clean-room LZO1X encoder from FableForge, so files the original game
must read no longer need GPL code to be compressed; upstream LZO stays an optional byte-exact
test. New fork formats use zstd/LZ4/zlib.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tructed raster with VS 2022

fable_title_view now has the fork's two planned backends (--api d3d12|vulkan) instead of D3D11;
their captured frames are pixel-identical. modern_title_raster.py compiles the parity lane's 17
reconstructed renderer sources and harness, unmodified, with VS 2022 x86: all 8 frames match
the VC7.1 build byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ical to VC7.1)

The parity lane's 17 reconstructed renderer sources build unmodified with VS 2022 x64 and draw the
retail title byte-identically through 64-bit D3D9; only three 32-bit layout shortcuts in the test
harness are patched, in generated copies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n tests

326 behaviour tests pass under VC7.1; 308 still pass when built with VS 2022 x86 and 179 with
VS 2022 x64. The report groups the rest by cause (bitset layout, asm bakes, 32-bit layouts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n the behaviour census

fable::compat::retail_bitset keeps VC7.1's 32-bit-word storage. With CThing::UsingComponents on it,
315 of the 326 VC7.1-passing reconstructed-function tests pass under VS 2022 x86 (was 308).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ayouts

96 of the 104 x64 runtime failures come from fixtures with 32-bit storage arrays or byte offsets,
not from the reconstructed code under test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… traces to a 32-bit test harness

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MouseAccumulator folds every OS mouse report into the next world tick's frame, so no polling rate
can overflow it; RawMouseWin32 feeds it from WM_INPUT. The injected bursts that overflowed
retail's 256-record DirectInput buffer are all counted (busiest tick 459 reports).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pdate

A world frame runs when the predicted present is more than one 15 Hz period past the current
frame; tests replay the live sample (4 renders per world frame at 60 Hz) and keep the world at
15 Hz at any refresh rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…DX's snapshot plus events

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…separate triggers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@BuffJesus
BuffJesus merged commit 78f6c75 into main Oct 24, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant