Repository navigation
Conversation
…ever dispatched (#1688) The argument names no CPU, and the key held only its text: two machines passing -march=native shared one key, so a shared cache could serve one CPU's object to another (an AVX-512 object on a machine without it dies with SIGILL). A dispatched compile was worse: the worker resolved native to its own CPU and the client stored that object under its key. The key now folds HostCpuKeyItem over the driver's -### listing of the unit's own line: gcc's expanded -march=<cpu>, feature switches and --param cache sizes, clang's -target-cpu, -tune-cpu and -target-feature values -- and none of the listing's paths, so the same CPU under another install prefix still shares. A listing naming no target keeps the compile uncached; a line naming native is refused for dispatch. Closes #1688 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…1689) -MD/-MMD alone make gcc and clang write the dependency record beside the object, and Kbuild's -Wp,-MMD,<file> names it outright, but the launcher knew a depfile only through -MF. A HIT therefore restored the object and no depfile, and a hand-written Makefile build (`$(CC) -MMD -MP -c $< -o $@`) lost every header dependency of that object: header edits stopped triggering rebuilds. Reproduced with the launcher. ParseCommand now derives the depfile the driver writes: the -o path with its last extension replaced by .d, or .d appended when there is none (measured on gcc and clang: a.o -> a.d, a.cpp.o -> a.cpp.d, noext -> noext.d), or the file -Wp,-M[M]D, names. -MF still names it outright. CMake builds always passed -MF and are unaffected. Closes #1689 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…ached (#1691) A hit restores the object and the depfile and nothing else, but several GNU flags make the compile write a second file: -gsplit-dwarf (.dwo), --coverage / -ftest-coverage (.gcno), clang -ftime-trace (.json), -fstack-usage (.su), -save-temps, the -fdump-* family, -fsave-optimization-record, -fcallgraph-info, -aux-info. After a hit the side file was missing, or left over from another build -- a .dwo whose id the restored skeleton does not match, a .gcno gcov rejects. Reproduced: the previous launcher cached -gsplit-dwarf and --coverage compiles. They are SideArtefacts rows now, so such a line is not cached and not dispatched, as the rule for second-artefact compiles already said. Closes #1691 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…irectory is not cached (#1692) These flags make the compiler read a file that reaches neither the preprocessed text nor the depfile, so the key could not see it change. Reproduced by the audit: gcc -fprofile-use with and without its .gcda gave identical -E text and depfiles and different objects -- one key, so every PGO build after a profile refresh was served the object built from the old profile. -fprofile-use, -fprofile-sample-use, -fauto-profile, -fprofile-instr-use, -fplugin, -fpass-plugin, -specs and -B are SideArtefacts rows now, beside cl's /Yu, which is there for the same reason. Generating a profile (-fprofile-generate) writes it at run time and stays cacheable. Hashing the named files instead would let PGO builds cache; refusing is the correct floor. Closes #1692 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…re their own outcomes (#1707) Three ways the statistics read healthy while the cache was not working: - A STORE that did not land (refused, or the exchange ran out of budget) was logged MISS, which the log defines as compiled AND stored -- observed while measuring #1686. It is UNSTORED now, with the cause in `detail`. - A hit refused as STALE (a replayed dependency missing here) was compiled and logged as an ordinary MISS, so a key stale on every build looked like a cold cache. It is STALE. - A FASTCACHE_VERIFY sample whose fresh compile FAILED was reported "inconclusive" and tallied as a HIT, though a source that does not compile has no object of its own and the one served for it is suspect. RestoreServedObject now answers FreshCompileFailed once the served object is back, which records VERIFY-FAILED and says on stderr that the entry may be stale; the build keeps the served object, as before. All three count as cacheable compiles in the hit rate and have their own report line and HTML card. Refs #1707 -- the verifier still compares only the object, not the depfile or streams, and the servers still count hits on values the launcher rejects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
… Open work Deferred work is an issue linked from the matching rulebook, never a residual in prose: the launcher findings go to compile-cache.md (#1693-#1700, #1707's remainder, #1708), the disk-tier ones to storage.md (#1690, #1702-#1705), and the protocol-surface and node ones to distributed-compilation.md (#1701, #1706). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The next batch from the compile-cache audit filed as #1688–#1708. Each change closes a way the cache served a wrong or incomplete result, or one way its reports hid that. Stacked on #1687 (based on its branch); it retargets to
masteronce #1687 merges.Changes
-march=nativeis keyed by the CPU it resolves to (closes fastcache-cc: -march=native is keyed as literal text, so one CPU's object is served to another #1688).-###listing of the unit's own line expandsnativeto: gcc's-march=<cpu>, feature switches and--paramcache sizes; clang's-target-cpu/-target-feature. None of the listing's paths go in.nativeis never dispatched, because the worker would resolve it to its own CPU.-MFis restored by a hit (closes fastcache-cc: -MD/-MMD without -MF -- a cache hit writes no depfile #1689).-MD/-MMDalone, and Kbuild's-Wp,-MMD,<file>, make the driver write a depfile the launcher did not know about. A hit restored the object and no depfile, so a Makefile build lost every header dependency.-opath with its last extension replaced by.d, as measured on gcc and clang.-gsplit-dwarf,--coverage/-ftest-coverage,-ftime-trace,-fstack-usage,-save-temps,-fdump-*,-fsave-optimization-record,-fcallgraph-info,-aux-info.-fprofile-useand the other profile-use flags, plugins,-specs=,-B.UNSTORED: the store did not land.STALE: a hit refused as stale and recompiled.VERIFY-FAILED: a sampled hit whose fresh compile failed. That case was reported "inconclusive" and tallied as a HIT.## Open work. fastcached disk tier: flush_all is undone by a restart #1690 (flush_allundone by a restart) needs a disk-format decision and is not in this PR.What is missing (why this is a draft)
The full build and all three unit suites pass locally (
FastCacheTest,fastcache-cc-tests,fastcache-compile-node-tests). Two of the 258 hygiene checks fail and are not fixed yet:test-name-hygiene: the new case"-march=native is keyed by the CPU the driver resolves it to, and by nothing else"insrc/apps/fastcache-cc/CacheKey_test.cppstarts with-, which Catch2 reads as an option. It needs renaming, e.g. "Naming -march=native keys the CPU the driver resolves it to".table-totals: a table in.agent/rules/distributed-compilation.md(line ~1104) has no<!-- table-total: ... -->marker. This check fails only on this branch, so the cause sits in this branch's change to that file; it is not yet narrowed down.Also not done:
flush_allundone by a restart) is open and needs a decision. Persisting the flush generation changes the disk format, so existing stores would need conversion; resetting the store file on flush is the alternative.🤖 Generated with Claude Code
https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU