Skip to content

Reuse cached objects across folders and machines - #1686

Merged
christianparpart merged 4 commits into
masterfrom
fix/test-checkout-paths-off-command-line
Oct 17, 2026
Merged

christianparpart merged 4 commits into
masterfrom
fix/test-checkout-paths-off-command-line

Conversation

@Yaraslaut

Copy link
Copy Markdown
Member

A second checkout of this repository on the same machine recompiled 359 of its 1017 units, and a second machine whose user has a different home directory recompiled 1145 of 1155. Every one of those misses was silent: the build succeeded, it simply was not shared. This branch makes both share, and makes any project that maps __FILE__ itself (Catch2 does) share across checkouts at all.

Measured on this repository (clang-release on the host, gcc-release in two Docker containers with different users, homes and checkout paths, all against one fastcached):

Second build Before After
Same machine, another folder 658 / 1017 hits 996 / 1017 hits (514 s → 23 s)
Another machine, CPM cache in its own home 10 / 1155 hits 1128 / 1155 hits (584 s → 36 s)

A further rebuild on the first machine with FASTCACHE_VERIFY=1 (every hit recompiled and compared) found no differing object among 1134 hits. On Catch2 v3.6.0 alone, a second folder went from 0 / 107 to 107 / 107 hits, each served object byte-identical to that folder's uncached compile.

Changes

  • The test targets' checkout paths leave the command line. FastCacheTest and fastcache-compile-node-tests passed FASTCACHED_SOURCE_DIR/FASTCACHED_TESTDATA_DIR as target-wide definitions, which put an absolute path into the key of all 382 of their units. They now come from a generated <tests/CheckoutPaths.hpp>, included by the 20 cases that read a file out of the checkout.
  • -ffile-prefix-map and -fmacro-prefix-map are keyed portably (PathValueRole::SourcePrefixMap). They reached the key verbatim, absolute root included. The head is relativized and the replacement kept literal, as for -fdebug-prefix-map. The flag stays on the preprocess line, because it rewrites the __FILE__ the key hashes. Where the build tree sits under the source root is folded into the key whenever such a map is present.
  • A dependency cache outside the checkout is a root of its own: <DEPROOT>, PathCanon::Layout::dependencyRoot, FASTCACHE_DEPS_DIR, which the CMake module fills from CPM_SOURCE_CACHE.
    • The key tokenizes it, and its headers count as content in the dependency set and the manifest.
    • An object naming it is bound to the machine (CheckoutPart::DependencyRoot).
    • The launcher tokenizes it in a value's text regions before the STORE, and a hit localizes it against its own root. Servers canonicalize with the two roots their frame carries, so their behaviour and CompileValueVersion do not move.
    • objkey-v8 / manifest-v9 keep the generations apart.
  • This repository maps __FILE__ for the checkout and the dependency cache, under fastcache-cc only (_fc_source_prefix_map_rules, checked by debug-prefix-map-rules). Before this, every Catch2 test object named its checkout and was stored per checkout.
  • -fmacro-prefix-map travels to a worker verbatim. It is a RemoteLine::Forwarded column, asked ahead of the role's refusal. The worker's allowlist gains an ArgValue::PrefixMapRule kind for a rule whose halves are paths it never opens. -ffile-prefix-map, which also rewrites debug records, still keeps the compile local.
  • Rules (compile-cache.md, AGENT.md tripwires) and the launcher, node and distributed-compilation docs describe all of the above.

Not covered

  • The 20 checkout-reading test units compile the checkout path in on purpose and stay bound to it.
  • Sources that sit outside the CMake source directory (lz4 here, zstd's build/cmake layout) still miss across machines. CMake embeds their absolute path in the object path, -MT and -MF, and the replayed depfile's target has to match.
  • Debug records of objects built from dependency sources still name the producing machine's cache. A dispatched compile carries one debug mapping pair, so a third rule would make local and dispatched objects differ in their debug records.
  • A failed STORE is still logged as MISS. Found while measuring; not touched here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU

Yaraslaut and others added 4 commits October 17, 2026 17:43
…r reuses the test targets' objects

FastCacheTest and fastcache-compile-node-tests passed the repository root (and the
fixtures directory) as target_compile_definitions, which put an absolute checkout path
on the command line of every unit in both targets -- 382 of a clang-release build's
1017. fastcache-cc keys on the arguments as well as on the preprocessed text, so each
of those units got a key no other checkout computes, and a second folder recompiled
all of them, most of which never read the path.

The paths now come from a generated <tests/CheckoutPaths.hpp>, included only by the
twenty cases that read a file out of the checkout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…ably, so a project mapping __FILE__ shares across checkouts

Both flags were unrecognised and reached the key verbatim, absolute checkout root
included. They are exactly what a project passes so its objects stop naming its
checkout -- Catch2's own CMake passes -ffile-prefix-map=<checkout>/= on every
compile -- so no two checkouts of such a project ever shared an object: 0 of 107
hits from a second folder.

They are now a path-value role of their own, SourcePrefixMap. The key relativizes
the head and leaves the replacement literal, as -fdebug-prefix-map's row does, and
the role differs from it in what reaching the preprocessed text requires: the flag
stays on the preprocess line, so the key hashes the __FILE__ the compile produces;
a line carrying one is never dispatched; and where the build tree sits under the
source root is folded into the key, because the map rewrites a build-tree path
through the source root's rule and a source_location there reaches no key text.

Measured: 107 of 107 hits from a second folder, every served object byte-identical
to that folder's own uncached compile. No objkey/manifest bump, by ComputeKey's v3
argument: one input stopped carrying a machine-specific string.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…maps __FILE__, so folders and machines share every object

Two things kept objects from being reused, both measured on this repository:

- Across MACHINES: every include directory into CPM's source cache, which lives under
  the user's home, reached the key verbatim. Two Docker machines with different homes
  shared 10 of 1155 objects; with the cache at one path, 794.
- Across FOLDERS and machines alike: every Catch2 test object names its checkout through
  __FILE__, so root binding stored it per checkout -- 358 of 1017 objects.

The dependency cache becomes a third root, <DEPROOT> (PathCanon::Layout::dependencyRoot,
FASTCACHE_DEPS_DIR). The key tokenizes it, the classifier counts its headers as content,
an object naming it is bound to the machine (CheckoutPart::DependencyRoot), and the
launcher tokenizes it in a value's text regions itself before the STORE -- a server knows
only the two roots its frame carries, so its canonicalization does not move, and the hit
localizes <DEPROOT> against its own root instead of replaying a depfile naming headers
that are not there. objkey-v8 and manifest-v9 keep the two generations apart.

The CMake module passes CPM_SOURCE_CACHE as FASTCACHE_DEPS_DIR (the CPM defaulting moves
ahead of it) and, under fastcache-cc alone, maps __FILE__ for the checkout and the cache
(_fc_source_prefix_map_rules, checked by debug-prefix-map-rules). -fmacro-prefix-map now
travels to a worker verbatim -- a RemoteLine column of the path-value table, and an
allowlist value kind for a rule whose halves are paths it never opens -- while
-ffile-prefix-map, which also rewrites debug records, keeps the compile local.

Not covered: debug records of objects built from dependency sources still name the
producing machine's cache, because a dispatched compile carries one debug mapping pair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…ne has

A real dependency root in an object's strings binds that object to the machine, so
the fixture's /home/bob/.cache/fastcached/cpm was a root-bound miss on exactly the
machine called bob in the cross-machine measurement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
@Yaraslaut Yaraslaut added the type/bug Behaves incorrectly against its stated contract label Oct 17, 2026
@github-actions github-actions Bot added area/launcher apps/fastcache-cc/ - the compiler launcher and its cache key area/compile-cache CompileCache/, the 0xFC wire, the executor, PathCanon area/distributed Distributed/, apps/fastcache-compile-node/ - dispatch, workers, scheduler area/net Net/, Async/, Server/ - sockets, reactors, connections area/platform Platform/, Config/, Cli/ - service registration, config lookup, the CLI table area/observability Metrics/, logging, stats and scrape surfaces area/build CMake, presets, toolchains, CI workflows area/core Core/ - the shared foundation; high blast radius labels Oct 17, 2026

@christianparpart christianparpart left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved while it's hot. Mind, I gonna cut a new release by tomorrow night, such that we hae a new one by Monday morning!

@christianparpart
christianparpart merged commit 5b019bc into master Oct 17, 2026
27 of 29 checks passed
@christianparpart
christianparpart deleted the fix/test-checkout-paths-off-command-line branch October 17, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build CMake, presets, toolchains, CI workflows area/compile-cache CompileCache/, the 0xFC wire, the executor, PathCanon area/core Core/ - the shared foundation; high blast radius area/distributed Distributed/, apps/fastcache-compile-node/ - dispatch, workers, scheduler area/launcher apps/fastcache-cc/ - the compiler launcher and its cache key area/net Net/, Async/, Server/ - sockets, reactors, connections area/observability Metrics/, logging, stats and scrape surfaces area/platform Platform/, Config/, Cli/ - service registration, config lookup, the CLI table type/bug Behaves incorrectly against its stated contract

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants