Skip to content

Bound the graph on disk, and let sites suppress their own navigation - #5

Merged
Bahbv merged 2 commits into
mainfrom
feat/graph-retention-and-nav-fanout
Sep 22, 2026
Merged

Bahbv merged 2 commits into
mainfrom
feat/graph-retention-and-nav-fanout

Conversation

@Bahbv

@Bahbv Bahbv commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Two problems found on a site running v2 since early September.

Graph growth

The graph only learns about a URL by rendering it, so nothing ever removed a URL that quietly fell out of the static cache. That site's graph described 53,713 URLs while only 5,880 were cached, and untracked() compared every save against rows that could never match.

Adds prune(array $keepUrls) and a cache-invalidation:prune command that feeds it the currently cached URLs. CachedUrls::supported() guards the case that would otherwise empty the graph — a cacher that cannot enumerate returns an empty list, which is indistinguishable from an empty cache.

Sqlite file size

DELETE moves pages onto sqlite's freelist and never shortens the file. The same site carried a 251 MB file whose freelist held 93% of its pages — roughly 18 MB of live rows. Flushing on StaticCacheCleared made it worse, not better.

Adds compact() to the contract: a real VACUUM on sqlite, a no-op where the database manages its own storage. Under WAL, VACUUM alone still leaves the main file at its old length, so it is followed by a truncating checkpoint. The test asserts the file actually shrinks, which is what caught that.

Navigation built by hand

TrackingNavTag already records {{ nav }} as the navigation rather than as every entry in it. A site that assembles its menu in its own PHP gets no such treatment, so each entry the menu reads becomes a dependency of every page carrying it.

On that site the seven top-level landing pages were each a dependency of 77–99% of all tracked URLs, which is why saving one of them timed the invalidation job out — every Invalidate entry in failed_jobs between 3 and 15 September was one of these.

suppressed() already existed for Statamic's own resolution machinery but was not reachable from outside. Exposed as CacheTags::withoutRecording(), to be paired with add() so one honest dependency stays behind:

CacheTags::add('nav:main');

$menu = CacheTags::withoutRecording(fn () => $this->buildMenu());

Applied on the reporting site, its navigation went from dozens of entry: tags to exactly five coarse ones, and none of the landing pages.

Notes

  • 156 tests, 266 assertions.
  • README gains two sections; CHANGELOG released as 2.0.1.
  • Strictly this is additive and could be 2.1.0; tagging 2.0.1 as requested. ^2.0 consumers get it either way.

…ation

Two problems found on a site running v2 since early September.

Graph growth
------------
The graph only learns about a URL by rendering it, so nothing ever removed a URL
that quietly fell out of the static cache. That site's graph described 53,713
URLs while only 5,880 were cached, and `untracked()` compared every save against
rows that could never match.

Adds `prune(array $keepUrls)` and a `cache-invalidation:prune` command that feeds
it the currently cached URLs. CachedUrls::supported() guards the case that would
otherwise empty the graph: a cacher that cannot enumerate returns an empty list,
which is indistinguishable from an empty cache.

Sqlite file size
----------------
`DELETE` moves pages onto sqlite's freelist and never shortens the file, so the
same site carried a 251 MB file whose freelist held 93% of its pages -- roughly
18 MB of live rows. Flushing on StaticCacheCleared made it worse, not better.

Adds `compact()` to the contract: a real VACUUM on sqlite, a no-op where the
database manages its own storage. Under WAL, VACUUM alone still leaves the main
file at its old length, so it is followed by a truncating checkpoint -- the test
asserts the file actually shrinks, which is what caught that.

Navigation built by hand
------------------------
TrackingNavTag already records `{{ nav }}` as the navigation rather than as every
entry in it. A site that assembles its menu in its own PHP gets no such
treatment, so each entry the menu reads becomes a dependency of every page
carrying it. On that site the seven top-level landing pages were each a
dependency of 77-99% of all tracked URLs, which is why saving one of them timed
the invalidation job out.

`suppressed()` already existed for Statamic's own resolution machinery but was
not reachable from outside. Exposes it as `CacheTags::withoutRecording()`, to be
paired with `add()` so one honest dependency stays behind.

156 tests, 266 assertions.
@Bahbv
Bahbv merged commit d1c4281 into main Sep 22, 2026
3 checks passed
@Bahbv
Bahbv deleted the feat/graph-retention-and-nav-fanout branch September 22, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant