perf(prefetch): finish the migration, delete the legacy helper, ratchet page graphs

Answers the question the previous commit left open: the tables list did not have
to stay on HTTP. lib/table/service reached the executor through
jobs/service -> rows/service -> workflow-columns, for one symbol.
pendingDeleteMask is a delete-visibility SQL clause with no executor
involvement, so it moves to its own leaf and that chain is cut. The tables
prefetch now reads the data layer like every other one, and
prefetch-internal-fetch.ts is deleted: nothing in the app calls its own API over
HTTP during a server render any more.

stripGroupDeps likewise moves to a leaf rather than being re-exported through
workflow-columns, so its importers no longer pull the executor to get a pure
projection.

React Query mechanism fixes, all found by audit:
- settings/[section] fired two prefetches without awaiting them. Only a settled
  query is dehydrated, so those were shipped mid-flight; a rejection hydrated
  into an error state retryOnMount: false never retries, leaving the panel
  broken for the session. Awaited now, and the pending-dehydration opt-in is
  removed since nothing streams.
- The viewer profile was prefetched by both the layout and the settings page.
  Separate server QueryClients mean that was a real second read per request.
- prefetchSubscriptionData was dead, and hand-rolled an unannotated raw fetch.
- retry is scoped to the browser. Query core defaults it to 0 on the server;
  stating one value for both opted awaited prefetches into a retry backoff. The
  gcTime default is dropped entirely — 5 minutes is already the browser default,
  and setting it explicitly overrode the server's Infinity, leaving a live timer
  and payload per request.

check:tool-registry-boundary now also ratchets per-page module counts against a
committed baseline, attributing a regression to the import that caused it via a
dominator tree. It caught a +444 regression in this branch by hand; it would
have caught it in CI. Its import regex also missed bare side-effect imports,
so `import '@/tools/registry'` could have slipped past it entirely.

Prefetch guidance added to .claude/rules/sim-queries.md.

Author: waleedlatif1

.agents/skills/tool-registry-boundary/SKILL.md

@@ -68,6 +68,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the
 
 Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph.
 
+The same command also ratchets those counts. `scripts/check-tool-registry-boundary.baseline.json` records each entry's module count plus its heaviest "gateway" modules (how many modules reach the graph *only* through each one). `--check` — what CI runs — fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, and names the gateway whose weight grew along with the import chain that reaches it. That catches the graph bloat the registry rule misses: the `listTables` prefetch that cost the Tables page 444 modules never touched `@/tools/registry`.
+
+When the growth is deliberate (a real new feature on the page), re-record it with `--update-baseline` and commit the JSON. When an entry *shrinks*, the check says so and passes — re-record then too, or the win is silently spendable again.
+
 ## How to verify an edge actually got cut
 
 Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph:

.claude/commands/tool-registry-boundary.md

@@ -67,6 +67,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the
 
 Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph.
 
+The same command also ratchets those counts. `scripts/check-tool-registry-boundary.baseline.json` records each entry's module count plus its heaviest "gateway" modules (how many modules reach the graph *only* through each one). `--check` — what CI runs — fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, and names the gateway whose weight grew along with the import chain that reaches it. That catches the graph bloat the registry rule misses: the `listTables` prefetch that cost the Tables page 444 modules never touched `@/tools/registry`.
+
+When the growth is deliberate (a real new feature on the page), re-record it with `--update-baseline` and commit the JSON. When an entry *shrinks*, the check says so and passes — re-record then too, or the win is silently spendable again.
+
 ## How to verify an edge actually got cut
 
 Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph:

.claude/rules/sim-queries.md

@@ -143,6 +143,22 @@ const handler = useCallback(() => {
 }, [data])
 ```
 
+## Server prefetching
+
+A server prefetch fills the *same* cache key a client hook fills, so it must be indistinguishable from a client fetch. Five rules:
+
+1. **Read the data layer, never our own API over HTTP.** A server-to-server call to `/api/...` costs a round trip and a second authentication for data the process can already read. Where the route runs an application use case, call that same use case with a principal from the same auth policy the route declares — not a manager underneath it.
+2. **Match the wire shape the hook caches.** The hook's data is whatever `requestJson(contract, …)` produced, so the seed must equal it. Two traps: a contract field declared `z.coerce.date()` means the hook holds a `Date` where raw route JSON holds a string; a passthrough response schema (`z.custom`) means the hook caches route JSON *verbatim*, so seeding raw rows leaks `Date`s and server-only fields. When the route projects before responding, share that projection — have the route and the prefetch call one function.
+3. **Prove the viewer.** Data-layer reads carry no authorization; the route used to provide it. Resolve the viewer (`getWorkspaceHostContextForViewer`, already `cache`d by the layout so it costs nothing) and return early on failure, caching nothing — the client fetch then reaches the route for the real 403. Never widen what a viewer can see.
+4. **Always `await`.** Only a settled query is dehydrated, so an unawaited prefetch is silently dropped from the payload and the pane waterfalls anyway.
+5. **Don't repeat what the layout already seeded.** `getQueryClient()` builds a new client per server call, so a page re-seeding a layout key is a genuine second read — and `HydrationBoundary` defers an already-seen query to an effect, which SSR never runs, so it never reaches the server render either.
+
+Reuse the hook's exported `staleTime` constant and its key factory; `dehydrate` carries neither options nor `staleTime`, and freshness is per-observer.
+
+Seed with `setQueryData` only when the prefetch must be able to *decline* to create an entry (an empty list that has to fall through to a route's creation path). `prefetchQuery` and `ensureQueryData` always create one.
+
+Keep prefetch imports light. A page prefetch's imports land in that route's server graph, so pulling a barrel to reach one function can drag thousands of modules behind it — `bun run check:tool-registry-boundary` gates this per page.
+
 ## Boundary Types
 
 - Hooks import named type aliases from `@/lib/api/contracts/**` (e.g., `import { listEntitiesContract, type EntityList } from '@/lib/api/contracts/entities'`). Never write `z.input<...>` / `z.output<...>` in hooks, and never `import { z } from 'zod'` in client code.

.cursor/commands/tool-registry-boundary.md

@@ -63,6 +63,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the
 
 Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph.
 
+The same command also ratchets those counts. `scripts/check-tool-registry-boundary.baseline.json` records each entry's module count plus its heaviest "gateway" modules (how many modules reach the graph *only* through each one). `--check` — what CI runs — fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, and names the gateway whose weight grew along with the import chain that reaches it. That catches the graph bloat the registry rule misses: the `listTables` prefetch that cost the Tables page 444 modules never touched `@/tools/registry`.
+
+When the growth is deliberate (a real new feature on the page), re-record it with `--update-baseline` and commit the JSON. When an entry *shrinks*, the check says so and passes — re-record then too, or the win is silently spendable again.
+
 ## How to verify an edge actually got cut
 
 Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph:

apps/sim/app/_shell/providers/get-query-client.ts

@@ -1,12 +1,11 @@
-import { defaultShouldDehydrateQuery, isServer, QueryClient } from '@tanstack/react-query'
+import { isServer, QueryClient } from '@tanstack/react-query'
 import { isDesktopApp } from '@/lib/desktop'
 
 export function makeQueryClient() {
   return new QueryClient({
     defaultOptions: {
       queries: {
         staleTime: 30 * 1000,
-        gcTime: 5 * 60 * 1000,
         // The desktop app window lives for days, so cross-session changes —
         // an admin upgrading your org/workspace role, a workspace you were
         // auto-added to, seat/entitlement changes — would otherwise stay
@@ -18,16 +17,19 @@ export function makeQueryClient() {
         // frequent and noisy. Per-query overrides (e.g. useWorkspaceSchedules
         // pins this off) always win over this default.
         refetchOnWindowFocus: isDesktopApp(),
-        retry: 1,
+        /**
+         * Query core already defaults retries to 0 on the server and 3 in the browser;
+         * only the browser number is ours to change. Stating one value for both would
+         * silently opt server prefetches into a retry, and because the layout awaits
+         * them that spends a retry backoff of document latency on a read whose failure
+         * the client recovers from on its own.
+         */
+        retry: isServer ? 0 : 1,
         retryOnMount: false,
       },
       mutations: {
         retry: false,
       },
-      dehydrate: {
-        shouldDehydrateQuery: (query) =>
-          defaultShouldDehydrateQuery(query) || query.state.status === 'pending',
-      },
     },
   })
 }

apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts

@@ -12,7 +12,7 @@ const {
   mockListFoldersForWorkspace,
   mockListInternalKnowledgeBases,
   mockListPinnedItemsForUser,
-  mockPrefetchInternalJson,
+  mockListTables,
   mockListWorkspaceFileFolders,
   mockListWorkspaceFilesWithShares,
 } = vi.hoisted(() => ({
@@ -23,7 +23,7 @@ const {
   mockListFoldersForWorkspace: vi.fn(),
   mockListInternalKnowledgeBases: vi.fn(),
   mockListPinnedItemsForUser: vi.fn(),
-  mockPrefetchInternalJson: vi.fn(),
+  mockListTables: vi.fn(),
   mockListWorkspaceFileFolders: vi.fn(),
   mockListWorkspaceFilesWithShares: vi.fn(),
 }))
@@ -46,8 +46,17 @@ vi.mock('@/lib/pinned-items/queries', () => ({
 vi.mock('@/lib/workspaces/permissions/utils', () => ({
   getWorkspaceMemberProfiles: mockGetWorkspaceMemberProfiles,
 }))
-vi.mock('@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch', () => ({
-  prefetchInternalJson: mockPrefetchInternalJson,
+vi.mock('@/lib/table/service', () => ({
+  listTables: mockListTables,
+}))
+/**
+ * `typeMetadataOf` is the one leaf of the real wire projection that reaches the
+ * column-type registry, and through it every type module's icon and editor. Stub
+ * that leaf only, so `toTableListItem`'s timestamp, `metadata`, and job
+ * normalization stay under test rather than being mocked away wholesale.
+ */
+vi.mock('@/lib/table/column-types', () => ({
+  typeMetadataOf: () => ({}),
 }))
 vi.mock('@/lib/api/server/routes', () => ({
   internalSessionAuth: { authenticate: mockAuthenticate },
@@ -90,7 +99,7 @@ describe('workspace list prefetches', () => {
     mockListWorkspaceFileFolders.mockResolvedValue([])
     mockListPinnedItemsForUser.mockResolvedValue([])
     mockGetWorkspaceMemberProfiles.mockResolvedValue([])
-    mockPrefetchInternalJson.mockResolvedValue({ data: { tables: [] } })
+    mockListTables.mockResolvedValue([])
     mockAuthenticate.mockResolvedValue({ kind: 'session', userId: USER_ID, sessionId: 'sess-1' })
     mockListInternalKnowledgeBases.mockResolvedValue({ knowledgeBases: [] })
     mockKnowledgePresenterList.mockReturnValue({ success: true, data: [] })
@@ -183,22 +192,67 @@ describe('workspace list prefetches', () => {
   })
 
   describe('prefetchTables', () => {
+    const TABLE_ROW = {
+      id: 't-1',
+      name: 'people',
+      description: null,
+      schema: { columns: [{ id: 'c1', name: 'name', type: 'string' }] },
+      metadata: { columnWidths: { c1: 120 } },
+      rowCount: 3,
+      maxRows: 10_000,
+      workspaceId: WORKSPACE_ID,
+      folderId: null,
+      createdBy: 'u-1',
+      locks: {
+        schemaLocked: false,
+        insertLocked: false,
+        updateLocked: false,
+        deleteLocked: false,
+      },
+      archivedAt: null,
+      createdAt: new Date('2026-01-01T00:00:00.000Z'),
+      updatedAt: new Date('2026-01-02T00:00:00.000Z'),
+    }
+
+    it('reads tables from the data layer', async () => {
+      mockListTables.mockResolvedValue([TABLE_ROW])
+      const client = makeClient()
+
+      await prefetchTables(client, WORKSPACE_ID, USER_ID)
+
+      expect(mockListTables).toHaveBeenCalledWith(WORKSPACE_ID, { scope: 'active' })
+    })
+
     /**
-     * The tables list is the one read on this page still served over HTTP: `listTables` lives in
-     * a module graph that reaches the executable tool registry, which
-     * `check:tool-registry-boundary` refuses to let into a page graph.
+     * `listTablesContract`'s response schema is a passthrough, so a client fetch caches the
+     * route's JSON verbatim. Seeding the raw data-layer row would put `Date`s and the
+     * server-only `metadata` field under a key the hook never sees them on.
      */
-    it('primes the exact key useTablesList reads and unwraps data.tables', async () => {
-      const tables = [{ id: 't-1' }]
-      mockPrefetchInternalJson.mockResolvedValue({ data: { tables } })
+    it('seeds the wire shape a client fetch caches, not the raw data-layer row', async () => {
+      mockListTables.mockResolvedValue([TABLE_ROW])
       const client = makeClient()
 
       await prefetchTables(client, WORKSPACE_ID, USER_ID)
 
-      expect(mockPrefetchInternalJson).toHaveBeenCalledWith(
-        `/api/table?workspaceId=${WORKSPACE_ID}&scope=active`
-      )
-      expect(client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active'))).toEqual(tables)
+      const [cached] = client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active')) as Array<
+        Record<string, unknown>
+      >
+      expect(cached.createdAt).toBe('2026-01-01T00:00:00.000Z')
+      expect(cached.updatedAt).toBe('2026-01-02T00:00:00.000Z')
+      expect(cached.archivedAt).toBeNull()
+      expect(cached).not.toHaveProperty('metadata')
+      expect(cached.jobStatus).toBeNull()
+      expect(cached.jobRowsProcessed).toBe(0)
+    })
+
+    it('caches no tables when the viewer cannot be proved', async () => {
+      mockGetWorkspaceHostContextForViewer.mockResolvedValue(null)
+      const client = makeClient()
+
+      await prefetchTables(client, WORKSPACE_ID, USER_ID)
+
+      expect(mockListTables).not.toHaveBeenCalled()
+      expect(client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active'))).toBeUndefined()
     })
   })
   describe('prefetchFilesBrowser', () => {
@@ -335,7 +389,7 @@ describe('workspace list prefetches', () => {
         const boom = new Error('500')
         mockListWorkspaceFilesWithShares.mockRejectedValue(boom)
         mockListFoldersForWorkspace.mockRejectedValue(boom)
-        mockPrefetchInternalJson.mockRejectedValue(boom)
+        mockListTables.mockRejectedValue(boom)
         mockListInternalKnowledgeBases.mockRejectedValue(boom)
         mockListPinnedItemsForUser.mockRejectedValue(boom)
         mockGetWorkspaceMemberProfiles.mockRejectedValue(boom)

apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts

@@ -120,12 +120,11 @@ async function seedWorkspaceList(
  * seeds nothing, leaving the client fetch to reach `GET /api/workspaces`'
  * default-workspace creation path — the same outcome a rejecting `queryFn` used
  * to produce, without routing a normal state through the error channel. That
- * matters because `makeQueryClient` dehydrates pending queries and sets
- * `retryOnMount: false`: were this read ever deferred, its rejection would
- * hydrate the client query into an error state nothing retries, permanently
- * locking a brand-new viewer out of workspace creation. Seeding also skips the
- * `retry: 1` default, which previously ran the whole read a second time, a
- * retry delay later, purely to re-derive an outcome already known.
+ * matters because only a settled query is dehydrated: an unawaited read would be
+ * dropped from the payload entirely, so the switcher would waterfall on every
+ * cold load rather than paint populated. Seeding also skips the `retry` default,
+ * which previously ran the whole read a second time, a retry delay later, purely
+ * to re-derive an outcome already known.
  */
 export async function prefetchWorkspaceSidebar(
   queryClient: QueryClient,

apps/sim/app/workspace/[workspaceId]/prefetch.ts

@@ -25,7 +25,7 @@ import {
 } from '@/app/workspace/[workspaceId]/settings/navigation'
 import { resolveWorkspaceGroup } from '@/ee/access-control/utils/permission-check'
 import { isForkingAvailableForWorkspace } from '@/ee/workspace-forking/lib/lineage/authz'
-import { prefetchGeneralSettings, prefetchUserProfile } from './prefetch'
+import { prefetchGeneralSettings } from './prefetch'
 import { SettingsPage } from './settings'
 
 interface WorkspaceSettingsSectionPageProps {
@@ -170,8 +170,14 @@ export default async function WorkspaceSettingsSectionPage({
   }
 
   const queryClient = getQueryClient()
-  void prefetchGeneralSettings(queryClient)
-  void prefetchUserProfile(queryClient)
+  /**
+   * Awaited, not fired and forgotten. An unawaited prefetch is still `pending` when
+   * `dehydrate` runs, so its rejection would hydrate the client query straight into an
+   * error state that `retryOnMount: false` never retries — leaving the panel broken for
+   * the rest of the session. The viewer's profile is already seeded by the workspace
+   * layout under the same key, so it is not repeated here.
+   */
+  await prefetchGeneralSettings(queryClient)
 
   return (
     <HydrationBoundary state={dehydrate(queryClient)}>