Repository navigation
Reuse cached objects across folders and machines - #1686
Merged
christianparpart merged 4 commits intoOct 17, 2026
Merged
Conversation
…r reuses the test targets' objects FastCacheTest and fastcache-compile-node-tests passed the repository root (and the fixtures directory) as target_compile_definitions, which put an absolute checkout path on the command line of every unit in both targets -- 382 of a clang-release build's 1017. fastcache-cc keys on the arguments as well as on the preprocessed text, so each of those units got a key no other checkout computes, and a second folder recompiled all of them, most of which never read the path. The paths now come from a generated <tests/CheckoutPaths.hpp>, included only by the twenty cases that read a file out of the checkout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…ably, so a project mapping __FILE__ shares across checkouts Both flags were unrecognised and reached the key verbatim, absolute checkout root included. They are exactly what a project passes so its objects stop naming its checkout -- Catch2's own CMake passes -ffile-prefix-map=<checkout>/= on every compile -- so no two checkouts of such a project ever shared an object: 0 of 107 hits from a second folder. They are now a path-value role of their own, SourcePrefixMap. The key relativizes the head and leaves the replacement literal, as -fdebug-prefix-map's row does, and the role differs from it in what reaching the preprocessed text requires: the flag stays on the preprocess line, so the key hashes the __FILE__ the compile produces; a line carrying one is never dispatched; and where the build tree sits under the source root is folded into the key, because the map rewrites a build-tree path through the source root's rule and a source_location there reaches no key text. Measured: 107 of 107 hits from a second folder, every served object byte-identical to that folder's own uncached compile. No objkey/manifest bump, by ComputeKey's v3 argument: one input stopped carrying a machine-specific string. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…maps __FILE__, so folders and machines share every object Two things kept objects from being reused, both measured on this repository: - Across MACHINES: every include directory into CPM's source cache, which lives under the user's home, reached the key verbatim. Two Docker machines with different homes shared 10 of 1155 objects; with the cache at one path, 794. - Across FOLDERS and machines alike: every Catch2 test object names its checkout through __FILE__, so root binding stored it per checkout -- 358 of 1017 objects. The dependency cache becomes a third root, <DEPROOT> (PathCanon::Layout::dependencyRoot, FASTCACHE_DEPS_DIR). The key tokenizes it, the classifier counts its headers as content, an object naming it is bound to the machine (CheckoutPart::DependencyRoot), and the launcher tokenizes it in a value's text regions itself before the STORE -- a server knows only the two roots its frame carries, so its canonicalization does not move, and the hit localizes <DEPROOT> against its own root instead of replaying a depfile naming headers that are not there. objkey-v8 and manifest-v9 keep the two generations apart. The CMake module passes CPM_SOURCE_CACHE as FASTCACHE_DEPS_DIR (the CPM defaulting moves ahead of it) and, under fastcache-cc alone, maps __FILE__ for the checkout and the cache (_fc_source_prefix_map_rules, checked by debug-prefix-map-rules). -fmacro-prefix-map now travels to a worker verbatim -- a RemoteLine column of the path-value table, and an allowlist value kind for a rule whose halves are paths it never opens -- while -ffile-prefix-map, which also rewrites debug records, keeps the compile local. Not covered: debug records of objects built from dependency sources still name the producing machine's cache, because a dispatched compile carries one debug mapping pair. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…ne has A real dependency root in an object's strings binds that object to the machine, so the fixture's /home/bob/.cache/fastcached/cpm was a root-bound miss on exactly the machine called bob in the cross-machine measurement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
christianparpart
approved these changes
Oct 17, 2026
christianparpart
left a comment
Member
There was a problem hiding this comment.
approved while it's hot. Mind, I gonna cut a new release by tomorrow night, such that we hae a new one by Monday morning!
This was referenced Oct 17, 2026
fastcache-cc: -march=native is keyed as literal text, so one CPU's object is served to another
#1688
Open
Open
This was referenced Oct 17, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A second checkout of this repository on the same machine recompiled 359 of its 1017 units, and a second machine whose user has a different home directory recompiled 1145 of 1155. Every one of those misses was silent: the build succeeded, it simply was not shared. This branch makes both share, and makes any project that maps
__FILE__itself (Catch2 does) share across checkouts at all.Measured on this repository (clang-release on the host, gcc-release in two Docker containers with different users, homes and checkout paths, all against one
fastcached):A further rebuild on the first machine with
FASTCACHE_VERIFY=1(every hit recompiled and compared) found no differing object among 1134 hits. On Catch2 v3.6.0 alone, a second folder went from 0 / 107 to 107 / 107 hits, each served object byte-identical to that folder's uncached compile.Changes
FastCacheTestandfastcache-compile-node-testspassedFASTCACHED_SOURCE_DIR/FASTCACHED_TESTDATA_DIRas target-wide definitions, which put an absolute path into the key of all 382 of their units. They now come from a generated<tests/CheckoutPaths.hpp>, included by the 20 cases that read a file out of the checkout.-ffile-prefix-mapand-fmacro-prefix-mapare keyed portably (PathValueRole::SourcePrefixMap). They reached the key verbatim, absolute root included. The head is relativized and the replacement kept literal, as for-fdebug-prefix-map. The flag stays on the preprocess line, because it rewrites the__FILE__the key hashes. Where the build tree sits under the source root is folded into the key whenever such a map is present.<DEPROOT>,PathCanon::Layout::dependencyRoot,FASTCACHE_DEPS_DIR, which the CMake module fills fromCPM_SOURCE_CACHE.CheckoutPart::DependencyRoot).CompileValueVersiondo not move.objkey-v8/manifest-v9keep the generations apart.__FILE__for the checkout and the dependency cache, underfastcache-cconly (_fc_source_prefix_map_rules, checked bydebug-prefix-map-rules). Before this, every Catch2 test object named its checkout and was stored per checkout.-fmacro-prefix-maptravels to a worker verbatim. It is aRemoteLine::Forwardedcolumn, asked ahead of the role's refusal. The worker's allowlist gains anArgValue::PrefixMapRulekind for a rule whose halves are paths it never opens.-ffile-prefix-map, which also rewrites debug records, still keeps the compile local.compile-cache.md,AGENT.mdtripwires) and the launcher, node and distributed-compilation docs describe all of the above.Not covered
build/cmakelayout) still miss across machines. CMake embeds their absolute path in the object path,-MTand-MF, and the replayed depfile's target has to match.MISS. Found while measuring; not touched here.🤖 Generated with Claude Code
https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU