How to set up an API-backed permanent store?

[Nick-Motion]

We're evaluating TanStack DB for a large application where we need to:

1. Fetch paginated data from an API (cursor based)
2. Store models in a normalized collection permanently (survive component unmounts)
3. Render filtered/sorted views over the accumulated data via useLiveQuery

After extensive testing and source code review, we've found that query collections cannot serve this use case due to three compounding issues:

1. Pagination is broken in on-demand mode (#820)

useLiveInfiniteQuery with syncMode: 'on-demand' doesn't implement cursor-based pagination. It appears to be designed only for moving a local window over a full dataset, rather than fetching pages on-demand.

The proposed pagination docs in #1355 (Approach 2, marked "Recommended") won't work because of this.

2. Row eviction on unsubscribe deletes permanent data (#1309)

When a component unmounts, subscription.unsubscribe() calls unloadSubset() for each loaded subset. The refcount in queryToRows/rowToQueries drops to 0, and cleanupQueryInternal deletes the rows from the collection via write({ type: 'delete' }).

gcTime does not prevent this — it only controls TanStack Query's cache lifetime, not the DB collection's row lifecycle.

3. Full State Sync prevents page accumulation

makeQueryResultHandler treats each queryFn result as complete state — items present in the old result but absent in the new result are deleted. This means anything written via writeInsert gets wiped on refetch of pages.

Questions:

1. Is there a planned path for "permanent collection" semantics — where rows survive unsubscribe and accumulate across paginated fetches?

2. What's the intended architecture for our use case? Using a Query Collection or Tanstack Query as the fetch/pagination layer and a separate local collection as the permanent store, and bridging them manually, creates a disconnect b/t the live query and the render - doesn't really work for making calls on-demand.

Versions:

    "@tanstack/db": "0.5.33",
    "@tanstack/query-db-collection": "1.0.30",
    "@tanstack/react-db": "0.1.77",

[RobinLbt]

Running into the same problem @Nick-Motion , did you end up with a specific solution?

[Nick-Motion]

After even more extensive testing and researching the library internals, I've come to the (surprising) conclusion that there is no supported solution for this. I ended up creating my own custom collection, which maintains a permanent model store, plus custom live-query hooks to manage. I've had to layer a lot of complexity on top of this, because the library doesn't seem to support this use case - the biggest problem being that there's a disconnect between "API request completed" and "Live query has materialized rows and is ready to render", meaning that isLoading cannot be relied upon in on-demand driven live queries.

I would love to be proven wrong here, or to have this use case supported officially.

[RobinLbt]

For context, I’m building app with user-defined saved views over a fairly normalized local data model

A simplified version of the canonical collections looks like this:

• threads
• thread_members
• people
• account_people
• person_labels
• labels

I originally tried to keep this fully canonical:

1. Fetch filtered/paginated thread IDs from the backend
2. Store only ordered IDs + cursor locally
3. Use live queries over the canonical collections to rebuild the visible rows
4. Apply saved-view filters across multiple related collections, including nested subqueries

Architecturally that felt right, but in practice it became very laggy once the local dataset got into the low-thousands of rows.

The problem was not one flat query, but the combination of:

• pagination
• user-defined filters
• multiple collection layers
• nested graph reconstruction from normalized entities

So I introduced a workaround:

1. The backend returns fully hydrated rows for a given saved-view page
2. I store those rows in dedicated localOnly collections that act as a read model for that view
3. Separately, I store query metadata such as:
   • ordered row_keys
   • cursor
   • counts
4. The UI renders directly from those hydrated local collections instead of reconstructing everything from canonical collections on demand

This gives much better first-render coherence and avoids the heavy lag from rebuilding the graph locally

The downside is that I now have a parallel read model, so canonical optimistic mutations also need to patch the hydrated view collections to keep them coherent. In other words, writes to the source-of-truth collections also have to maintain the read model

My questions for @KyleAMathews are:

1. Is this kind of dedicated localOnly read-model collection an acceptable pattern in TanStack DB?
   Or is the intended approach to keep everything in canonical collections and make the live-query path efficient enough for this sort of large paginated filtered view?
   More broadly, are read-only/materialized collections for large inbox/search/list views considered normal in TanStack DB, or more of an anti-pattern?