Skip to content

fix: normalize keys before join operation - #1229

Merged
kevin-dp merged 7 commits into
TanStack:mainfrom
hugiex:fix/join-date-eq-comparison
Feb 12, 2026
Merged

kevin-dp merged 7 commits into
TanStack:mainfrom
hugiex:fix/join-date-eq-comparison

Conversation

@hugiex

@hugiex hugiex commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

Description

To fix the issue #934

Problem

  • join conditions like eq(left.date, right.date) were using raw Date object as join keys
  • the join index uses Map key identify => the diff Date instance with the same timestamp did not match

Solution

  • normalize join keys (Date -> timestamp)
  • also tests created to cover the initial matching and live updates when timestamps changes

@changeset-bot

changeset-bot Bot commented Feb 10, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 5bcb579

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages
Name Type
@tanstack/db Patch
@tanstack/angular-db Patch
@tanstack/electric-db-collection Patch
@tanstack/offline-transactions Patch
@tanstack/powersync-db-collection Patch
@tanstack/query-db-collection Patch
@tanstack/react-db Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Feb 11, 2026

Copy link
Copy Markdown
More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1229

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1229

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1229

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1229

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1229

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1229

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1229

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1229

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1229

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1229

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1229

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1229

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1229

commit: 3a23b30

…tion

The join lazy-loading path passes normalized join keys (e.g. Date→timestamp)
into `inArray`, but the `in` evaluator compared raw field values against
them using `Array.includes` (strict equality). This meant the optimization
silently fell back to loading the full collection for Date-keyed joins.

Normalize the value in the `in` evaluator, consistent with how `eq` already
normalizes both operands.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@kevin-dp

Copy link
Copy Markdown
Contributor

Thanks for the fix! I pushed an additional commit to also normalize the value in the in evaluator (evaluators.ts).

Why this is needed: The join compiler has a lazy-loading optimization where the active side's join keys are passed into inArray(ref, joinKeys) to selectively load matching rows from the lazy side (instead of loading the entire collection). After your fix, those joinKeys are normalized (e.g. Date → timestamp number), but the in evaluator was comparing them against the raw field values using Array.includes() (strict equality). So [1234567890].includes(new Date(1234567890)) would return false, causing the optimization to silently fall back to loading the full collection.

The fix normalizes the value in the in evaluator, consistent with how eq already normalizes both operands.

kevin-dp and others added 2 commits February 12, 2026 14:34
Move the two Date join tests from the separate join-date.test.ts file
into the existing Complex Join Scenarios block in join.test.ts, keeping
all join tests in one place.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
kevin-dp and others added 2 commits February 12, 2026 14:48
Adds a test verifying that inArray correctly filters rows by Date field
values when the array contains Date objects.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The `in` evaluator used `array.includes(value)` which relies on strict
equality. When either the value or the array elements are Date objects
(or other types that normalizeValue handles), reference equality fails
even when the underlying values are the same.

Normalize array elements via `array.some(item => normalizeValue(item) === value)`
so that both sides are compared as primitives, consistent with how `eq`
already normalizes both operands.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@kevin-dp

kevin-dp commented Feb 12, 2026

Copy link
Copy Markdown
Contributor

While reviewing, I found an additional related bug: inArray in WHERE clauses also doesn't work with Date objects. Reproduced in this test: https://github.com/TanStack/db/actions/runs/21949227368/job/63395185329

For example, .where(({ item }) => inArray(item.createdAt, [new Date('2025-01-01')])) returns no results because the in evaluator uses Array.includes() which relies on strict reference equality (===), so different Date instances with the same timestamp never match.

This is the same class of bug as the original join issue — just in a different code path. The fix (commit 3a23b30) normalizes both the value and the array elements in the in evaluator:

// Before
return array.includes(value)

// After
return array.some((item) => normalizeValue(item) === value)

This makes in consistent with eq, which already normalizes both operands. A test was added in the previous commit (f57873c) to cover this case.

@samwillis samwillis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@kevin-dp
kevin-dp merged commit 83d5ac8 into TanStack:main Feb 12, 2026
@github-actions github-actions Bot mentioned this pull request Feb 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This PR has been released!

Thank you for your contribution!

KyleAMathews added a commit that referenced this pull request Sep 16, 2026
Canonicalize satisfiable join operands through the compilation ValueIdentity, keep nullish operands row-disjoint, and preserve raw lazy-demand values with stable representatives. This restores equality-predicate consistency for binary and other established value domains without adding compound join syntax.

Provenance: #593 Lucas Duailibe and Sam Willis; #861 Vincent Chan; #896 Tomas Zaluckij; #779/#899/#1258 Kyle Mathews and Claude; #899 review by Sam Willis; #1229 Hieu Nguyen, Hiếu Nguyễn Minh, Kevin De Porre, and Claude Opus 4.6; #1797 ValueIdentity architecture by Kyle Mathews. Prior art informed the evidence and design; no prior implementation lines were reused. #593/#861 remain design-gated.

Weight: production source net +30 lines; emitted join module +65 B ESM gzip and +60 B CJS gzip. No public API, export, dependency, compatibility branch, or db-ivm change.
KyleAMathews added a commit that referenced this pull request Sep 16, 2026
Canonicalize satisfiable join operands through the compilation ValueIdentity, keep nullish operands row-disjoint, and preserve raw lazy-demand values with stable representatives. This restores equality-predicate consistency for binary and other established value domains without adding compound join syntax.

Provenance: #593 Lucas Duailibe and Sam Willis; #861 Vincent Chan; #896 Tomas Zaluckij; #779/#899/#1258 Kyle Mathews and Claude; #899 review by Sam Willis; #1229 Hieu Nguyen, Hiếu Nguyễn Minh, Kevin De Porre, and Claude Opus 4.6; #1797 ValueIdentity architecture by Kyle Mathews. Prior art informed the evidence and design; no prior implementation lines were reused. #593/#861 remain design-gated.

Weight: production source net +30 lines; emitted join module +65 B ESM gzip and +60 B CJS gzip. No public API, export, dependency, compatibility branch, or db-ivm change.
KyleAMathews added a commit that referenced this pull request Sep 17, 2026
* fix(db): align join keys with equality identity

Canonicalize satisfiable join operands through the compilation ValueIdentity, keep nullish operands row-disjoint, and preserve raw lazy-demand values with stable representatives. This restores equality-predicate consistency for binary and other established value domains without adding compound join syntax.

Provenance: #593 Lucas Duailibe and Sam Willis; #861 Vincent Chan; #896 Tomas Zaluckij; #779/#899/#1258 Kyle Mathews and Claude; #899 review by Sam Willis; #1229 Hieu Nguyen, Hiếu Nguyễn Minh, Kevin De Porre, and Claude Opus 4.6; #1797 ValueIdentity architecture by Kyle Mathews. Prior art informed the evidence and design; no prior implementation lines were reused. #593/#861 remain design-gated.

Weight: production source net +30 lines; emitted join module +65 B ESM gzip and +60 B CJS gzip. No public API, export, dependency, compatibility branch, or db-ivm change.

* Apply join equality review fixes

* Reduce join result processing overhead
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants