fix(query-db-collection): implement reference counting for QueryObserver lifecycle

[KyleAMathews]

Summary

Fixes data loss on component remount by implementing reference counting infrastructure for QueryObserver lifecycle management.

The Problem (vs main)

When live query subscriptions unsubscribed, there was no tracking of which rows were still needed by other active queries. This caused:

• Data loss during component remounts
• Issues when multiple live queries shared identical predicates
• No way to properly clean up rows when last subscriber left

The Solution

Implemented reference counting to track QueryObserver usage:

1. Pass same predicates to unloadSubset that were passed to loadSubset
2. Use them to compute the queryKey (via generateQueryKeyFromOptions)
3. Use existing machinery (queryToRows map) to find rows that query loaded
4. Decrement the ref count
5. GC rows where count reaches 0 (no longer referenced by any active query)

What Changed

New Infrastructure (packages/db)

• Added UnloadSubsetFn type to sync config
• Collections now call unloadSubset when subscriptions clean up
• Subscriptions track all loadSubset calls to unload them symmetrically

Core Implementation (packages/query-db-collection)

• Reference counting map tracks live queries per QueryObserver
• generateQueryKeyFromOptions() ensures loadSubset/unloadSubset use identical keys
• Refcount increments when queries load/reuse observers
• Refcount decrements via unloadSubset when subscriptions clean up
• Row-level GC when refcount reaches 0
• Preserves TanStack Query's cache (no removeQueries or observer.destroy)

Impact

• ✅ Navigation back to previously loaded pages shows cached data immediately
• ✅ No unnecessary refetches during quick remounts (< gcTime)
• ✅ Multiple live queries with identical predicates correctly share QueryObservers
• ✅ Proper row-level cleanup when last subscriber leaves
• ✅ TanStack Query's cache lifecycle (gcTime, staleTime) is fully respected
• ✅ No data leakage from in-flight requests when unsubscribing

Test Coverage

Added comprehensive tests for QueryObserver lifecycle:

• Reference counting with duplicate subset loads
• Refcount reset after TanStack Query GC and reload
• Mount/unmount/remount without breaking cache
• No data leakage when unsubscribing during in-flight load

All 20 tests in "Query Garbage Collection" suite pass.

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 0917f9e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@tanstack/query-db-collection	Patch
@tanstack/db	Patch
@tanstack/db-collection-e2e	Patch
@tanstack/angular-db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/powersync-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/rxdb-db-collection	Patch
@tanstack/solid-db	Patch
@tanstack/svelte-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

More templates

• todos
• tanstack-start-example-basic
• @tanstack/db-example-paced-mutations-demo
• tanstack-start-db-projects-example
• @tanstack/db-example-react-todo
• @tanstack/db-example-solid-todo

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@870

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@870

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@870

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@870

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@870

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@870

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@870

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@870

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@870

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@870

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@870

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@870

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@870

commit: 0917f9e

[github-actions]

Size Change: +141 B (+0.16%)

Total Size: 86.3 kB

Filename	Size	Change
./packages/db/dist/esm/collection/subscription.js	2.48 kB	+64 B (+2.65%)
./packages/db/dist/esm/collection/sync.js	2.37 kB	+77 B (+3.36%)

ℹ️ View Unchanged

Filename	Size
./packages/db/dist/esm/collection/change-events.js	1.38 kB
./packages/db/dist/esm/collection/changes.js	977 B
./packages/db/dist/esm/collection/events.js	388 B
./packages/db/dist/esm/collection/index.js	3.24 kB
./packages/db/dist/esm/collection/indexes.js	1.1 kB
./packages/db/dist/esm/collection/lifecycle.js	1.67 kB
./packages/db/dist/esm/collection/mutations.js	2.26 kB
./packages/db/dist/esm/collection/state.js	3.43 kB
./packages/db/dist/esm/deferred.js	207 B
./packages/db/dist/esm/errors.js	4.19 kB
./packages/db/dist/esm/event-emitter.js	748 B
./packages/db/dist/esm/index.js	2.64 kB
./packages/db/dist/esm/indexes/auto-index.js	742 B
./packages/db/dist/esm/indexes/base-index.js	766 B
./packages/db/dist/esm/indexes/btree-index.js	1.87 kB
./packages/db/dist/esm/indexes/lazy-index.js	1.1 kB
./packages/db/dist/esm/indexes/reverse-index.js	513 B
./packages/db/dist/esm/local-only.js	837 B
./packages/db/dist/esm/local-storage.js	2.1 kB
./packages/db/dist/esm/optimistic-action.js	359 B
./packages/db/dist/esm/paced-mutations.js	496 B
./packages/db/dist/esm/proxy.js	3.22 kB
./packages/db/dist/esm/query/builder/functions.js	733 B
./packages/db/dist/esm/query/builder/index.js	3.96 kB
./packages/db/dist/esm/query/builder/ref-proxy.js	917 B
./packages/db/dist/esm/query/compiler/evaluators.js	1.35 kB
./packages/db/dist/esm/query/compiler/expressions.js	430 B
./packages/db/dist/esm/query/compiler/group-by.js	1.8 kB
./packages/db/dist/esm/query/compiler/index.js	1.96 kB
./packages/db/dist/esm/query/compiler/joins.js	2 kB
./packages/db/dist/esm/query/compiler/order-by.js	1.25 kB
./packages/db/dist/esm/query/compiler/select.js	1.07 kB
./packages/db/dist/esm/query/expression-helpers.js	1.43 kB
./packages/db/dist/esm/query/ir.js	673 B
./packages/db/dist/esm/query/live-query-collection.js	360 B
./packages/db/dist/esm/query/live/collection-config-builder.js	5.26 kB
./packages/db/dist/esm/query/live/collection-registry.js	264 B
./packages/db/dist/esm/query/live/collection-subscriber.js	1.74 kB
./packages/db/dist/esm/query/live/internal.js	130 B
./packages/db/dist/esm/query/optimizer.js	2.56 kB
./packages/db/dist/esm/query/predicate-utils.js	2.88 kB
./packages/db/dist/esm/query/subset-dedupe.js	921 B
./packages/db/dist/esm/scheduler.js	1.21 kB
./packages/db/dist/esm/SortedMap.js	1.18 kB
./packages/db/dist/esm/strategies/debounceStrategy.js	247 B
./packages/db/dist/esm/strategies/queueStrategy.js	428 B
./packages/db/dist/esm/strategies/throttleStrategy.js	246 B
./packages/db/dist/esm/transactions.js	2.9 kB
./packages/db/dist/esm/utils.js	881 B
./packages/db/dist/esm/utils/browser-polyfills.js	304 B
./packages/db/dist/esm/utils/btree.js	5.61 kB
./packages/db/dist/esm/utils/comparison.js	852 B
./packages/db/dist/esm/utils/index-optimization.js	1.51 kB
./packages/db/dist/esm/utils/type-guards.js	157 B

_{compressed-size-action::db-package-size}

[github-actions]

Size Change: 0 B

Total Size: 3.34 kB

ℹ️ View Unchanged

Filename	Size
./packages/react-db/dist/esm/index.js	225 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js	1.17 kB
./packages/react-db/dist/esm/useLiveQuery.js	1.11 kB
./packages/react-db/dist/esm/useLiveSuspenseQuery.js	431 B
./packages/react-db/dist/esm/usePacedMutations.js	401 B

_{compressed-size-action::react-db-package-size}

[samwillis]

LGTM

[samwillis]

Although having said that, there isn't a test. May be a good idea to add one?

[samwillis]

After the refactor to introduce unloadSubset my ✅ stands.

This works well for the queryCollection where there is a way to go from predicates to queryKey - we should also explore how we can use it to GC rows on an electric collection.

I suspect a similar snapshotKey that is uses to ref count loaded rows.

I wander if the key generation, and then row refcounting can become a genric we can reuse for both (and other) collections.

(obviously once CI is green)

[kevin-dp]

I'm not a fan of storing the LoadSubsetOptions for all loadSubset calls that we make because:

• i think most collections do some form of hashing/computing an internal key from these options and store that key internally. Both loadSubset and unloadSubset then perform this hashing.
• some collections don't use them in unloadSubset (e.g. Electric) which means wasted memory

I believe a more traditional approach is to let the callee (optionally) return an identifier that it can use to trace requests (in this case loadSubset calls). So i'd rather have loadSubset return an opaque identifier and the subscription only storing those identifiers. In unloadSubset we then pass that identifier and it is up to the collection to decide how to handle it. E.g. the query collection would return the query key, and then in unloadSubset it would get that query key and immediately proceed to decrementing the corresponding ref counts without having to recompute the query key from the options. For the Electric collection, since we don't GC (yet) its loadSubset could always return null meaning we only store 1 null value even if the subscription makes a lot of loadSubset calls.

And if a collection's unloadSubset would really need to know which options were passed initially, it can always track that internally: generate the opaque identifier and then internally map that identifier to the options.

[kevin-dp]

why have it as | null instead of:

public syncUnloadSubsetFn?: (options: LoadSubsetOptions) => void

An optional property is easier to handle than one that can be null. For example on L215, you would not have to do ?? null.

[kevin-dp]

You don't have that problem if you use identifiers to relate loadSubset and unloadSubset calls

[kevin-dp]

Based on the PR description, it seems that there were some problems/bugs with the existing ref counting implementation. However, this PR introduces a new ref counting mechanism while also keeping the old one but removing critical parts that break the old ref counting implementation. I don't think that introducing this new ref counting mechanism is the way to solve the problem. I'd rather keep the old ref counting system and fix the bugs it has.

The old ref counting system worked like this in a nutshell:

• Keeps track of which rows are referenced by which queries (based on query key)
• Keeps track of which queries (based on query key) reference which rows
• When a live query is GCed, the subscription is unsubscribed and we tell ts/query to GC all the query keys belonging to this subscription.
• When a ts/query is GCed we drop the ref count of all the rows it referenced
• If the ref count of a row hits zero we GC it

Iiuc, the bug you're describing in this PR is that several live queries may call loadSubset with the same predicate so they map to the same query key. But then when one of them is GCed, it would GC the ts/query even though the other one is still referencing it. I think the only thing we need to do to fix the existing implementation is to keep some additional state which for each query key keeps track which subscriptions are relying on that query, i.e. const querySubscribers: Map<QueryKey, Set<SubscriberId>>. Then, when modify the current flow like this (changes are in bolt):

• Keeps track of which rows are referenced by which queries (based on query key)
• Keeps track of which queries (based on query key) reference which rows
• Keeps track of which queries are referenced by which subscribers: const querySubscribers: Map<QueryKey, Set<SubscriberId>>
• When a live query is GCed, the subscription is unsubscribed and for all query keys used by this subscription we remove the subscription from querySubscribers. When a querySubscribers set for a certain query key becomes empty we tell ts/query to GC that query key.
• When a ts/query is GCed we drop the ref count of all the rows it referenced
• If the ref count of a row hits zero we GC it