Skip to content

Fix cache results for -march=native, implied depfiles, side artefacts and unseen inputs - #1711

Draft
Yaraslaut wants to merge 6 commits into
fix/key-line-structure-and-response-filesfrom
fix/cache-audit-native-depfile-flush
Draft

Yaraslaut wants to merge 6 commits into
fix/key-line-structure-and-response-filesfrom
fix/cache-audit-native-depfile-flush

Conversation

@Yaraslaut

Copy link
Copy Markdown
Member

The next batch from the compile-cache audit filed as #1688–#1708. Each change closes a way the cache served a wrong or incomplete result, or one way its reports hid that. Stacked on #1687 (based on its branch); it retargets to master once #1687 merges.

Changes

What is missing (why this is a draft)

The full build and all three unit suites pass locally (FastCacheTest, fastcache-cc-tests, fastcache-compile-node-tests). Two of the 258 hygiene checks fail and are not fixed yet:

  • test-name-hygiene: the new case "-march=native is keyed by the CPU the driver resolves it to, and by nothing else" in src/apps/fastcache-cc/CacheKey_test.cpp starts with -, which Catch2 reads as an option. It needs renaming, e.g. "Naming -march=native keys the CPU the driver resolves it to".
  • table-totals: a table in .agent/rules/distributed-compilation.md (line ~1104) has no <!-- table-total: ... --> marker. This check fails only on this branch, so the cause sits in this branch's change to that file; it is not yet narrowed down.

Also not done:

🤖 Generated with Claude Code

https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU

Yaraslaut and others added 6 commits October 17, 2026 19:07
…ever dispatched (#1688)

The argument names no CPU, and the key held only its text: two machines passing
-march=native shared one key, so a shared cache could serve one CPU's object to
another (an AVX-512 object on a machine without it dies with SIGILL). A dispatched
compile was worse: the worker resolved native to its own CPU and the client stored that
object under its key.

The key now folds HostCpuKeyItem over the driver's -### listing of the unit's own line:
gcc's expanded -march=<cpu>, feature switches and --param cache sizes, clang's
-target-cpu, -tune-cpu and -target-feature values -- and none of the listing's paths, so
the same CPU under another install prefix still shares. A listing naming no target keeps
the compile uncached; a line naming native is refused for dispatch.

Closes #1688

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…1689)

-MD/-MMD alone make gcc and clang write the dependency record beside the object, and
Kbuild's -Wp,-MMD,<file> names it outright, but the launcher knew a depfile only
through -MF. A HIT therefore restored the object and no depfile, and a hand-written
Makefile build (`$(CC) -MMD -MP -c $< -o $@`) lost every header dependency of that
object: header edits stopped triggering rebuilds. Reproduced with the launcher.

ParseCommand now derives the depfile the driver writes: the -o path with its last
extension replaced by .d, or .d appended when there is none (measured on gcc and clang:
a.o -> a.d, a.cpp.o -> a.cpp.d, noext -> noext.d), or the file -Wp,-M[M]D, names. -MF
still names it outright. CMake builds always passed -MF and are unaffected.

Closes #1689

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…ached (#1691)

A hit restores the object and the depfile and nothing else, but several GNU flags make
the compile write a second file: -gsplit-dwarf (.dwo), --coverage / -ftest-coverage
(.gcno), clang -ftime-trace (.json), -fstack-usage (.su), -save-temps, the -fdump-*
family, -fsave-optimization-record, -fcallgraph-info, -aux-info. After a hit the side
file was missing, or left over from another build -- a .dwo whose id the restored
skeleton does not match, a .gcno gcov rejects. Reproduced: the previous launcher cached
-gsplit-dwarf and --coverage compiles.

They are SideArtefacts rows now, so such a line is not cached and not dispatched, as the
rule for second-artefact compiles already said.

Closes #1691

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…irectory is not cached (#1692)

These flags make the compiler read a file that reaches neither the preprocessed text
nor the depfile, so the key could not see it change. Reproduced by the audit: gcc
-fprofile-use with and without its .gcda gave identical -E text and depfiles and
different objects -- one key, so every PGO build after a profile refresh was served the
object built from the old profile.

-fprofile-use, -fprofile-sample-use, -fauto-profile, -fprofile-instr-use, -fplugin,
-fpass-plugin, -specs and -B are SideArtefacts rows now, beside cl's /Yu, which is there
for the same reason. Generating a profile (-fprofile-generate) writes it at run time and
stays cacheable. Hashing the named files instead would let PGO builds cache; refusing is
the correct floor.

Closes #1692

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
…re their own outcomes (#1707)

Three ways the statistics read healthy while the cache was not working:

- A STORE that did not land (refused, or the exchange ran out of budget) was logged
  MISS, which the log defines as compiled AND stored -- observed while measuring #1686.
  It is UNSTORED now, with the cause in `detail`.
- A hit refused as STALE (a replayed dependency missing here) was compiled and logged as
  an ordinary MISS, so a key stale on every build looked like a cold cache. It is STALE.
- A FASTCACHE_VERIFY sample whose fresh compile FAILED was reported "inconclusive" and
  tallied as a HIT, though a source that does not compile has no object of its own and
  the one served for it is suspect. RestoreServedObject now answers FreshCompileFailed
  once the served object is back, which records VERIFY-FAILED and says on stderr that the
  entry may be stale; the build keeps the served object, as before.

All three count as cacheable compiles in the hit rate and have their own report line and
HTML card.

Refs #1707 -- the verifier still compares only the object, not the depfile or streams,
and the servers still count hits on values the launcher rejects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
… Open work

Deferred work is an issue linked from the matching rulebook, never a residual in prose:
the launcher findings go to compile-cache.md (#1693-#1700, #1707's remainder, #1708),
the disk-tier ones to storage.md (#1690, #1702-#1705), and the protocol-surface and node
ones to distributed-compilation.md (#1701, #1706).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnCkwgtUsaSM5HcgB3APNU
@Yaraslaut Yaraslaut added the type/bug Behaves incorrectly against its stated contract label Oct 17, 2026
@github-actions github-actions Bot added the area/launcher apps/fastcache-cc/ - the compiler launcher and its cache key label Oct 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/launcher apps/fastcache-cc/ - the compiler launcher and its cache key type/bug Behaves incorrectly against its stated contract

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant