feat: populate Workers KV with pre-rendered pages at deploy time by NathanDrake2406 · Pull Request #670 · cloudflare/vinext

NathanDrake2406 · 2026-03-23T05:44:21Z

Summary

Closes #562. Depends on #653 — requires buildId, trailingSlash, and router fields in the prerender manifest. Should be merged after #653.

New module cloudflare/populate-kv.ts — reads prerender manifest + HTML/RSC files, constructs cache entries matching the runtime format exactly, uploads via Cloudflare REST bulk API
Deploy integration — automatic step 6d between prerender asset copy and wrangler deploy when CLOUDFLARE_API_TOKEN + VINEXT_CACHE KV namespace + prerender manifest all exist. --no-populate-kv to opt out.
TPR refactor — replaces TPR's inline uploadToKV (80 lines) with shared helpers, fixing three bugs: wrong key format (cache:<path> → cache:app:<buildId>:<path>:html), wrong TTL (10x revalidate → fixed 30-day), missing tags (empty [] → full hierarchy for revalidatePath())

Key format parity

Deploy-time keys match the runtime __isrCacheKey in app-rsc-entry.ts exactly:

[appPrefix:]cache:app:<buildId>:<pathname>:<suffix>

Two entries per route (:html + :rsc) with full tag hierarchy (_N_T_/layout, intermediate layouts, leaf page tag) so revalidatePath() works on seeded entries.

Design decisions

App Router only — Pages Router uses different keys (pages:...) and value shape (kind: "PAGES"). Routes without router: "app" are skipped.
Batching — by count (10,000 max) and bytes (95 MB with 5 MB headroom from 100 MB API limit)
Fixed 30-day KV TTL — matches KVCacheHandler.set() default
No --app-prefix flag — removed after review; it would write prefixed keys without wiring the prefix into the generated runtime, creating silently unreadable entries. appPrefix remains on the programmatic PopulateKVOptions for advanced users who configure both sides.

Test plan

40 unit tests covering key parity, tag parity, entry serialization, bulk batching, end-to-end flow, router filtering, graceful skips
vp check passes (0 warnings, 0 errors)
Manual: CLOUDFLARE_API_TOKEN=... vinext deploy --prerender-all on a test project with ISR routes
Verify KV entries via wrangler kv key list --binding VINEXT_CACHE
First request shows X-Vinext-Cache: HIT

Add populate-kv.ts with cache key construction, tag generation, entry serialization, bulk upload, and end-to-end populateKV orchestrator. Key format matches runtime __isrCacheKey exactly: [appPrefix:]cache:app:<buildId>:<pathname>:<suffix> Produces two entries per route (:html + :rsc) with full tag hierarchy for revalidatePath() support. Uploads via Cloudflare REST bulk API with count-based (10k) and byte-based (95MB) batching. 38 tests covering key parity, tag parity, entry shape, batching, and end-to-end flow with temp directory fixtures.

pkg-pr-new · 2026-03-23T05:44:58Z

Open in StackBlitz

npm i https://pkg.pr.new/vinext@670

commit: 92d84d3

Add step 6c between prerender/TPR and wrangler deploy. Automatically populates KV when CLOUDFLARE_API_TOKEN + VINEXT_CACHE KV namespace + prerender manifest all exist. Silent skip when prerequisites missing. New CLI flags: --no-populate-kv Disable automatic KV population --app-prefix KVCacheHandler appPrefix for key construction Export resolveAccountId from tpr.ts for shared use.

Replace TPR's inline uploadToKV with shared buildRouteEntries and uploadBulkToKV from populate-kv.ts. This fixes three TPR bugs: - Key format: was cache:<path>, now cache:app:<buildId>:<path>:html - TTL: was 10x revalidate clamped, now fixed 30-day (matching runtime) - Tags: was empty [], now full hierarchy for revalidatePath() support BuildId is resolved from vinext-prerender.json when available.

Replace `as string` casts with type-safe parseFetchBody helper. Reorder imports: builtins → framework → project modules.

- Only seed routes with router: "app" — Pages Router uses different keys (pages:...) and value shape (kind: "PAGES"), so seeding them as APP_PAGE entries would be unreachable at runtime. - Routes without a router field are skipped defensively (pre-cloudflare#653 manifests lack this field, but also lack buildId so populateKV would already skip). - Remove --app-prefix CLI flag — it writes prefixed keys without wiring the same prefix into the generated runtime, creating silently unreadable cache entries. Keep appPrefix on the programmatic PopulateKVOptions for advanced users who configure both sides manually.

james-elicx · 2026-03-23T10:48:14Z

packages/vinext/src/deploy.ts

+  // Step 6c: Populate KV with pre-rendered pages
+  if (!options.noPopulateKv) {
+    const apiToken = process.env.CLOUDFLARE_API_TOKEN;
+    const wranglerConfig = parseWranglerConfig(root);
+    const kvNamespaceId = wranglerConfig?.kvNamespaceId;
+
+    if (apiToken && kvNamespaceId) {
+      const accountId = wranglerConfig?.accountId ?? (await resolveAccountId(apiToken));
+
+      if (accountId) {
+        const kvResult = await populateKV({
+          root,
+          accountId,
+          namespaceId: kvNamespaceId,
+          apiToken,
+        });
+
+        if (kvResult.skipped) {
+          console.log(`\n  KV populate: Skipped (${kvResult.skipped})`);
+        } else if (kvResult.entriesUploaded > 0) {
+          console.log(
+            `\n  KV populate: ${kvResult.routesProcessed} routes → ${kvResult.entriesUploaded} entries (${(kvResult.durationMs / 1000).toFixed(1)}s)`,
+          );
+        }
+      }
+    }
+  }


I think we need to better think about the API for this before implementing it.

There shouldn't be an implicit assumption that KV is the cache handler people use.

I have been thinking about this over the weekend - we need to ensure the solution we arrive at will scale to any cache handler rather than being KV-specific.

I'm not still not 100% on what that should look like. Some things I was thinking about at the weekend were having a static property on the cache handler itself that can define a method for build/deploy-time bulk-population, and then provide the handler using the next.config.js cacheHandler option.

That could work, but I think it might have an issue with bundling dependencies that we don't want bundled if it were to use wrangler for the population logic for example.

Lets continue a conversation about how this can be implemented in a way that scales to any cache handler on the issue before iterating on impl

Sorry for zoomin! I'll refactor it out of deploy and turn this into a KV adapter instead. Do you think the foundation part is good enough tho?

Yeah i think a config is best here. I'll get back to u w the details. Need to nerd out a bit

Lets discuss on the issue what good looks like for a long-term solution 👍

Lets discuss on the issue what good looks like for a long-term solution 👍

instead of a static method, what about a named export? It feels cleaner and more modular, less coupled.
I can refactor this PR to use the seeding core and the KV part to a KV adapter

As the decision we make will have to be supported for a long time ideally without needing breaking changes, it would probably be good for us to come up with a few different options that we can weigh up pros/cons for and hopefully get some engagement from others in the community on, with a caveat that whatever approach we take should still be fully compatible with the cache handler Next.js config option.

Yeah I think you should open an issue and pin it, maybe a discussion?

There's a placeholder issue for remote cache population - we can use that one perhaps?

NathanDrake2406 added 4 commits March 23, 2026 16:45

refactor: remove as casts and clean up test imports

754a7f9

Replace `as string` casts with type-safe parseFetchBody helper. Reorder imports: builtins → framework → project modules.

NathanDrake2406 force-pushed the feat/populate-kv-deploy branch from 32e6c66 to 68f3762 Compare March 23, 2026 05:45

NathanDrake2406 added 2 commits March 23, 2026 16:51

refactor: hoist invariant URL and headers out of upload loop

e8fb96d

ci: retrigger

92d84d3

james-elicx requested changes Mar 23, 2026

View reviewed changes

james-elicx marked this pull request as draft March 23, 2026 12:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: populate Workers KV with pre-rendered pages at deploy time#670

feat: populate Workers KV with pre-rendered pages at deploy time#670
NathanDrake2406 wants to merge 7 commits intocloudflare:mainfrom
NathanDrake2406:feat/populate-kv-deploy

NathanDrake2406 commented Mar 23, 2026 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Mar 23, 2026 •

edited

Loading

Uh oh!

james-elicx Mar 23, 2026

Uh oh!

NathanDrake2406 Mar 23, 2026

Uh oh!

NathanDrake2406 Mar 23, 2026

Uh oh!

james-elicx Mar 23, 2026

Uh oh!

NathanDrake2406 Mar 23, 2026

Uh oh!

james-elicx Mar 23, 2026 •

edited

Loading

Uh oh!

NathanDrake2406 Mar 23, 2026

Uh oh!

james-elicx Mar 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

NathanDrake2406 commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key format parity

Design decisions

Test plan

Uh oh!

pkg-pr-new bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

james-elicx Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

NathanDrake2406 Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

NathanDrake2406 Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

james-elicx Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

NathanDrake2406 Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

james-elicx Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NathanDrake2406 Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

james-elicx Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

NathanDrake2406 commented Mar 23, 2026 •

edited

Loading

pkg-pr-new bot commented Mar 23, 2026 •

edited

Loading

james-elicx Mar 23, 2026 •

edited

Loading