Skip to content

fix(core): make note_types search filter case-insensitive#912

Merged
phernandez merged 1 commit into
mainfrom
fix/note-types-case-insensitive
Jun 8, 2026
Merged

fix(core): make note_types search filter case-insensitive#912
phernandez merged 1 commit into
mainfrom
fix/note-types-case-insensitive

Conversation

@phernandez

Copy link
Copy Markdown
Member

Summary

The search_notes note_types filter was case-sensitive on both the SQLite and Postgres backends, despite the MCP tool documenting it as case-insensitive. The tool lowercases the input ("Chapter" -> "chapter"), but the stored frontmatter type value preserves its original casing ("Chapter"), so the SQL comparison never matched capitalized types. A note with type: Chapter was findable by plain text search but invisible to search_notes(note_types=["Chapter"]).

This fix folds case on both ends of the comparison.

Bugs fixed

  • Bug feat: Add new canvas tool to create json canvas files in obsidian. #14note_types filter is case-sensitive despite documented case-insensitivity (capitalized frontmatter type values were unfindable).
    • SQLite (sqlite_search_repository.py): wrap the column in LOWER(json_extract(metadata, '$.note_type')) and lowercase the parameter values.
    • Postgres (postgres_search_repository.py): replace the JSONB @> containment check (exact-match) with LOWER(metadata->>'note_type') IN (...) and lowercase the parameter values.
    • Values remain parameterized, so the existing SQL-injection protection is unchanged.

Testing

Test came from the integration bug hunt; added under test-int/mcp/test_search_note_types_case_insensitive.py (real DB / ASGI / MCP client, no mocks).

  • uv run pytest test-int/mcp/test_search_note_types_case_insensitive.py -q --no-cov -> 2 passed (SQLite)
  • BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest test-int/mcp/test_search_note_types_case_insensitive.py -q --no-cov -> 2 passed (Postgres)
  • uv run pytest tests/repository/test_search_repository.py test-int/mcp/test_search_integration.py -q --no-cov -> 50 passed (SQLite)
  • BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest tests/repository/test_postgres_search_repository.py tests/repository/test_search_repository.py -q --no-cov -> 47 passed, 12 skipped (Postgres)
  • uv run ruff check (scoped files) / uv run ruff format . / uv run ty check src tests test-int -> all pass

Risk

Low. The change only affects the note_types filter clause. Lowercasing the stored value at comparison time broadens matching to be case-insensitive (the documented behavior); the SQL-injection regression tests for note_types still pass on both backends. No tool signature changed.

🤖 Generated with Claude Code

Bug #14 (integration bug hunt): the search_notes note_types filter was
case-sensitive on both backends. The MCP tool documents note_types as
case-insensitive and lowercases the input ('Chapter' -> 'chapter'), but
the stored frontmatter type preserves its casing ('Chapter'), so the SQL
comparison never matched capitalized types.

Fix folds case on both ends:
- SQLite: LOWER(json_extract(metadata, '$.note_type')) IN (...) with
  lowercased param values.
- Postgres: replace JSONB '@>' containment with
  LOWER(metadata->>'note_type') IN (...) with lowercased param values.

Values stay parameterized, so SQL-injection protection is unchanged.

Adds the regression test from the integration bug hunt under
test-int/mcp/test_search_note_types_case_insensitive.py (real DB/ASGI/MCP
client, no mocks): a capitalized 'type: Chapter' note is now matched by
note_types=['Chapter'], plus a lowercase control.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Claude finished @phernandez's task in 1m 52s —— View job


Code Review — PR #912: fix(core): make note_types search filter case-insensitive

  • Read changed files and gather context
  • Review code quality and standards
  • Review testing coverage
  • Review security and performance
  • Post final review

Summary

This is a clean, minimal fix for a real bug. The root cause is correct: the SQLite backend's json_extract(...) IN (...) and the Postgres backend's JSONB @> containment were both exact-match comparisons, while the documented behavior and MCP tool layer both expect case-insensitive matching. Wrapping the stored value in LOWER() and lowercasing the filter parameters fixes the mismatch on both backends.


Checklist

Code Quality & Standards

  • Follows Basic Memory's coding conventions — parameter names, loop structure, and style all match the adjacent categories filter block
  • Python 3.12+ type annotations and async patterns — no changes to signatures
  • SQLAlchemy 2.0 best practices — continues using parameterized text() queries
  • 100-character line length limit — longest new line in SQLite file is ~67 chars; Postgres line is within limit
  • No commented-out code blocks
  • Literate programming style — new comments follow the required Trigger/Why/Outcome pattern

Testing & Documentation

  • Integration tests added (test-int/mcp/test_search_note_types_case_insensitive.py)
  • Control test verifies the previously-working lowercase path still works (no regression)
  • Tests use real DB + real ASGI + MCP client — no mocks
  • Missing test case: neither test covers note_types=["chapter"] (lowercase filter) against a note with type: Chapter (uppercase stored). The main test uses note_types=["Chapter"], which the MCP tool lowercases to "chapter" before calling the repo — but the test title "filter is case-insensitive" implies the filter value casing varies. Adding this explicit case removes ambiguity about what "case-insensitive" means.
  • No unit test in tests/repository/: the changed code is in _build_fts_query_parts. The integration tests provide full coverage at the MCP level, but a unit test for the repository query-building step (like the existing ones in tests/repository/test_search_repository.py) would be closer to the changed code and easier to read as documentation. Not a blocking concern if 100% coverage is already met by the integration path.

Basic Memory Architecture

  • No schema changes (no Alembic migration needed)
  • Local-first architecture preserved
  • Minimal diff — only the note_types filter clause changed, nothing else

Security & Performance

  • SQL injection protection preserved — values are still bound as named parameters (:note_type_0, :note_type_1, etc.)
  • No hardcoded secrets or credentials
  • The json.dumps removal in Postgres is safe — the old {"note_type": note_type} JSON-document containment pattern is replaced with a scalar extraction, which is simpler and equally correct
  • Minor performance note: LOWER(json_extract(...)) and LOWER(metadata->>'note_type') are not index-sargable, so a future index on note_type would not be used by this filter. Since metadata is a JSON/JSONB column with no current index on note_type, this is a non-issue in practice — but worth keeping in mind if note_types filtering becomes a hot path.

Compatibility

  • No file path comparisons involved
  • No emojis or non-ASCII characters added

Specific Code Notes

SQLite (sqlite_search_repository.py:819–828): The change from

params[param_name] = t
# ...
f"json_extract(search_index.metadata, '$.note_type') IN ({placeholders})"

to

params[param_name] = t.lower()
# ...
"LOWER(json_extract(search_index.metadata, '$.note_type')) IN ({placeholders})"

is correct and matches the pattern of the categories filter immediately above it.

Postgres (postgres_search_repository.py:787–795): The switch from JSONB @> containment (exact-match on a JSON document) to scalar text extraction + LOWER() + IN is the right approach. The previous code required json.dumps({"note_type": note_type}) to construct a containment document — the new pattern is simpler and semantically clearer. The json import is still used elsewhere in the file (json.loads in the result-mapping loop), so no unused import is created.

Behavior symmetry: Both backends now lowercase the stored value at query time via LOWER() and lowercase the filter values at bind time via .lower(). This means any combination of casing in the filter or stored frontmatter will match, which is the documented intent.


Verdict

Approve with minor suggestions. The fix is correct, the SQL injection protection is intact, and the integration tests validate both the fixed case and the regression guard. The two items above (additional test for lowercase-filter-vs-uppercase-stored, and a repository-layer unit test) would improve test documentation but are not blockers for merging.

@phernandez phernandez merged commit df7452e into main Jun 8, 2026
25 checks passed
@phernandez phernandez deleted the fix/note-types-case-insensitive branch June 8, 2026 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant