Skip to content

test: add result-level ANN plan validation above k threshold - #269

Merged
tickernelz merged 1 commit into
mainfrom
test/ann-result-level-validation
Aug 25, 2026
Merged

test: add result-level ANN plan validation above k threshold#269
tickernelz merged 1 commit into
mainfrom
test/ann-result-level-validation

Conversation

@lindixu6-hash

Copy link
Copy Markdown
Collaborator

Summary

Follow-up from #259 review feedback by @NaNomicon: the existing SQL-text assertion in turso-vector-search.test.ts only verifies the query contains CROSS JOIN and string ordering, which would pass even if the planner silently reordered and search degraded.

This adds a behavior-level test:

  • Seeds 200 real memories across two container tags (150 target + 50 other), well above the k=128 ANN threshold for tagged searches.
  • The target memory at index 0 has the query vector itself → must rank first with similarity > 0.9.
  • Verifies no other-container-tag memories leak into results (indices 150–199 must not appear).
  • Verifies result count respects the requested limit.

Testing

  • bun test tests/turso-vector-search.test.ts — 3/3 pass (existing 2 + new 1)
  • tsc --noEmit clean

Follow-up from #259 review: the existing SQL-text assertion only
verifies the query contains CROSS JOIN, which would pass even if the
planner silently reordered. This test seeds 200 real memories across two
container tags (above the k=128 ANN threshold), runs a tagged search,
and asserts that:
- the closest target-tag memory ranks first (similarity > 0.9)
- no other-container-tag memories leak into results
- the result count respects the limit

This catches future planner/driver regressions at the behavior level
rather than SQL string matching.
@lindixu6-hash

Copy link
Copy Markdown
Collaborator Author

Reviewer quick guide: This is a test-only PR (no source changes). It adds one integration test that seeds 200 real memories across two container tags (above the k=128 ANN threshold) and verifies:

  1. The closest target-tag memory ranks first (similarity > 0.9)
  2. No other-tag memories leak into results
  3. Result count respects the limit

This addresses @NaNomicon's follow-up from #259 review: the existing SQL-text test only checks for CROSS JOIN in the query string, which would pass even if the planner reordered. This test validates actual ANN behavior at the result level.

@tickernelz
tickernelz merged commit b3fe705 into main Aug 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants