fix: deduplicate and filter null join keys in lazy join subset queries by KyleAMathews · Pull Request #1448 · TanStack/db

KyleAMathews · 2026-04-03T14:56:04Z

Summary

Fix bloated ANY() SQL params in lazy join subset queries. When multiple rows reference the same foreign key or have null foreign keys, the join key array now gets deduplicated and null-filtered before being sent to Electric.

Root Cause

In joins.ts, the lazy join path collects foreign keys from all rows via .map(([[joinKey]]) => joinKey) and passes them directly to inArray(). With real data where multiple rows share a foreign key (e.g., 5 tasks in the same project) or have nullable foreign keys, the resulting ANY() param contains repeated IDs and NULLs:

WHERE "id" = ANY('{NULL,NULL,"proj-1","proj-1","proj-1","proj-2","proj-2"}')

Approach

Three-line fix at the collection point (joins.ts:305-317):

.filter((key) => key != null) — remove null/undefined keys (loose equality catches both)
new Set(...) — deduplicate remaining keys
Early return when no valid keys remain — skip the snapshot request entirely

Key Invariants

Join semantics are unchanged: left joins still produce undefined for unmatched rows
The early return on empty keys prevents pointless inArray(ref, []) calls to the sync layer
Set equality works correctly for the primitive types used as join keys (strings, numbers)

Non-goals

The pre-existing silent deoptimization fallback (lines 325-328) where a failed index lookup loads the entire collection was not addressed — that's a separate concern

Verification

pnpm -C packages/db exec vitest run -t "lazy join key deduplication"  # 2 new tests
pnpm -C packages/db exec vitest run                                   # full suite: 2256 passed
pnpm -C packages/db exec tsc --noEmit                                 # clean

Files Changed

File	Change
`packages/db/src/query/compiler/joins.ts`	Dedup + null-filter join keys, early return on empty
`packages/db/tests/query/live-query-collection.test.ts`	2 tests: mixed keys (dedup+nulls) and all-null keys (early return)
`.changeset/fix-lazy-join-key-dedup.md`	Changeset for patch release

🤖 Generated with Claude Code

…ests When a lazy join collected foreign keys for subset queries, duplicate IDs and null values were passed through to the ANY() SQL param, producing bloated queries. Filter nulls, deduplicate via Set, and skip the request entirely when no valid keys remain. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Both branches added new test describe blocks at the end of live-query-collection.test.ts — kept both. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

pkg-pr-new · 2026-04-03T15:07:32Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1448

@tanstack/browser-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1448

@tanstack/capacitor-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1448

@tanstack/cloudflare-durable-objects-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1448

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1448

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1448

@tanstack/db-sqlite-persistence-core

npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1448

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1448

@tanstack/electron-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1448

@tanstack/expo-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1448

@tanstack/node-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1448

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1448

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1448

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1448

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1448

@tanstack/react-native-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1448

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1448

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1448

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1448

@tanstack/tauri-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1448

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1448

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1448

commit: 39bdc41

github-actions · 2026-04-03T15:08:06Z

Size Change: +40 B (+0.04%)

Total Size: 113 kB

📦 View Changed

Filename	Size	Change
`packages/db/dist/esm/query/compiler/joins.js`	2.15 kB	+40 B (+1.9%)

ℹ️ View Unchanged

Filename	Size
`packages/db/dist/esm/collection/change-events.js`	1.39 kB
`packages/db/dist/esm/collection/changes.js`	1.38 kB
`packages/db/dist/esm/collection/cleanup-queue.js`	810 B
`packages/db/dist/esm/collection/events.js`	434 B
`packages/db/dist/esm/collection/index.js`	3.61 kB
`packages/db/dist/esm/collection/indexes.js`	1.99 kB
`packages/db/dist/esm/collection/lifecycle.js`	1.69 kB
`packages/db/dist/esm/collection/mutations.js`	2.47 kB
`packages/db/dist/esm/collection/state.js`	5.26 kB
`packages/db/dist/esm/collection/subscription.js`	3.74 kB
`packages/db/dist/esm/collection/sync.js`	2.88 kB
`packages/db/dist/esm/collection/transaction-metadata.js`	144 B
`packages/db/dist/esm/deferred.js`	207 B
`packages/db/dist/esm/errors.js`	4.92 kB
`packages/db/dist/esm/event-emitter.js`	748 B
`packages/db/dist/esm/index.js`	3 kB
`packages/db/dist/esm/indexes/auto-index.js`	830 B
`packages/db/dist/esm/indexes/base-index.js`	729 B
`packages/db/dist/esm/indexes/basic-index.js`	2.05 kB
`packages/db/dist/esm/indexes/btree-index.js`	2.17 kB
`packages/db/dist/esm/indexes/index-registry.js`	820 B
`packages/db/dist/esm/indexes/reverse-index.js`	538 B
`packages/db/dist/esm/local-only.js`	890 B
`packages/db/dist/esm/local-storage.js`	2.1 kB
`packages/db/dist/esm/optimistic-action.js`	359 B
`packages/db/dist/esm/paced-mutations.js`	496 B
`packages/db/dist/esm/proxy.js`	3.75 kB
`packages/db/dist/esm/query/builder/functions.js`	905 B
`packages/db/dist/esm/query/builder/index.js`	5.25 kB
`packages/db/dist/esm/query/builder/ref-proxy.js`	1.05 kB
`packages/db/dist/esm/query/compiler/evaluators.js`	1.62 kB
`packages/db/dist/esm/query/compiler/expressions.js`	430 B
`packages/db/dist/esm/query/compiler/group-by.js`	2.69 kB
`packages/db/dist/esm/query/compiler/index.js`	3.63 kB
`packages/db/dist/esm/query/compiler/order-by.js`	1.51 kB
`packages/db/dist/esm/query/compiler/select.js`	1.11 kB
`packages/db/dist/esm/query/effect.js`	4.78 kB
`packages/db/dist/esm/query/expression-helpers.js`	1.43 kB
`packages/db/dist/esm/query/ir.js`	829 B
`packages/db/dist/esm/query/live-query-collection.js`	360 B
`packages/db/dist/esm/query/live/collection-config-builder.js`	7.79 kB
`packages/db/dist/esm/query/live/collection-registry.js`	264 B
`packages/db/dist/esm/query/live/collection-subscriber.js`	1.94 kB
`packages/db/dist/esm/query/live/internal.js`	145 B
`packages/db/dist/esm/query/live/utils.js`	1.64 kB
`packages/db/dist/esm/query/optimizer.js`	2.62 kB
`packages/db/dist/esm/query/predicate-utils.js`	2.97 kB
`packages/db/dist/esm/query/query-once.js`	359 B
`packages/db/dist/esm/query/subset-dedupe.js`	960 B
`packages/db/dist/esm/scheduler.js`	1.3 kB
`packages/db/dist/esm/SortedMap.js`	1.3 kB
`packages/db/dist/esm/strategies/debounceStrategy.js`	247 B
`packages/db/dist/esm/strategies/queueStrategy.js`	428 B
`packages/db/dist/esm/strategies/throttleStrategy.js`	246 B
`packages/db/dist/esm/transactions.js`	2.9 kB
`packages/db/dist/esm/utils.js`	927 B
`packages/db/dist/esm/utils/array-utils.js`	273 B
`packages/db/dist/esm/utils/browser-polyfills.js`	304 B
`packages/db/dist/esm/utils/btree.js`	5.61 kB
`packages/db/dist/esm/utils/comparison.js`	1.05 kB
`packages/db/dist/esm/utils/cursor.js`	457 B
`packages/db/dist/esm/utils/index-optimization.js`	1.54 kB
`packages/db/dist/esm/utils/type-guards.js`	157 B
`packages/db/dist/esm/virtual-props.js`	360 B

_{compressed-size-action::db-package-size}

github-actions · 2026-04-03T15:08:40Z

Size Change: 0 B

Total Size: 4.24 kB

ℹ️ View Unchanged

Filename	Size
`packages/react-db/dist/esm/index.js`	249 B
`packages/react-db/dist/esm/useLiveInfiniteQuery.js`	1.32 kB
`packages/react-db/dist/esm/useLiveQuery.js`	1.34 kB
`packages/react-db/dist/esm/useLiveQueryEffect.js`	355 B
`packages/react-db/dist/esm/useLiveSuspenseQuery.js`	567 B
`packages/react-db/dist/esm/usePacedMutations.js`	401 B

_{compressed-size-action::react-db-package-size}

Assert capturedOptions is empty rather than checking for absence of inArray expressions. This catches regressions where the fallback path issues an expensive full collection load with no where clause. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

samwillis

KyleAMathews and others added 4 commits April 3, 2026 08:38

chore: add changeset for lazy join key dedup fix

ad13046

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

merge: resolve conflict with origin/main

32244c8

Both branches added new test describe blocks at the end of live-query-collection.test.ts — kept both. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ci: apply automated fixes

f6e7038

samwillis approved these changes Apr 3, 2026

View reviewed changes

KyleAMathews merged commit 3fe689a into main Apr 3, 2026
7 checks passed

KyleAMathews deleted the fix/lazy-join-key-dedup branch April 3, 2026 15:48

github-actions bot mentioned this pull request Mar 31, 2026

ci: Version Packages #1433

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: deduplicate and filter null join keys in lazy join subset queries#1448

fix: deduplicate and filter null join keys in lazy join subset queries#1448
KyleAMathews merged 5 commits intomainfrom
fix/lazy-join-key-dedup

KyleAMathews commented Apr 3, 2026

Uh oh!

pkg-pr-new bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

samwillis left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

KyleAMathews commented Apr 3, 2026

Summary

Root Cause

Approach

Key Invariants

Non-goals

Verification

Files Changed

Uh oh!

pkg-pr-new bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samwillis left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pkg-pr-new bot commented Apr 3, 2026 •

edited

Loading

github-actions bot commented Apr 3, 2026 •

edited

Loading

github-actions bot commented Apr 3, 2026 •

edited

Loading