Fast artifact export raw-copies a live WAL-mode DB — torn graph.db.zst that import then installs unchecked

[kriswill]

Version: codebase-memory-mcp 0.8.1 (built from source)
Platform: macOS (Apple Silicon) · Install channel: Built from source · Binary variant: standard

What happened, and what did you expect?

cbm_artifact_export with CBM_ARTIFACT_FAST (the watcher/incremental auto-update path in pipeline_incremental.c, whose return code is also ignored) reads the raw bytes of the main .db file via read_file_alloc. The store runs in WAL mode, so the main file alone misses committed transactions still in the -wal and can be mid-checkpoint — the copy is a torn snapshot.

A field artifact produced this way fails PRAGMA quick_check with four short indexes and btreeInitPage() returns error code 11 on ~12 nodes-table pages; its artifact.json compression_level: 3 fingerprints the fast path:

wrong # of entries in index idx_nodes_label
wrong # of entries in index idx_nodes_name
wrong # of entries in index idx_nodes_file
wrong # of entries in index sqlite_autoindex_nodes_1
Tree 21 page 8..19: btreeInitPage() returns error code 11   (Tree 21 = nodes table)

cbm_artifact_import then installs the torn artifact byte-for-byte as the live cache DB: cbm_store_check_integrity only sanity-checks the projects table, so page-level corruption passes.

Expected: exports snapshot consistently (VACUUM INTO, as the BEST path already does — the fast path can keep zstd-3 and skip the index-stripping), and import refuses files that fail quick_check.

Reproduction

1. Open a store and insert ~100 rows without closing — the rows are committed but live only in the -wal, not the main file bytes.
2. Call cbm_artifact_export(db_path, repo, project, CBM_ARTIFACT_FAST) while that connection is open (exactly what the watcher path does against the long-lived server's store).
3. cbm_artifact_import the artifact to a fresh path: the imported DB has the pre-WAL row count; under concurrent checkpointing the copy is torn at the page level as in the log above.

Project scale (field occurrence)

3,977 nodes / 6,666 edges / ~480 files

Cascade

The torn artifact this produces is rendered invisible by the scan-error swallowing bug (companion issue, linked in a comment below), which is how it served wrong answers in the field for weeks. Related same-class issues filed alongside: stale-WAL replay on DB swap, and malformed-JSON emission.

Maintainer note

I have a working private fix for this (with reproduce-first regression tests) that I can submit as a PR. Before I do, I'd prefer you groom this issue — confirm the framing, tag, and prioritize — so the PR lands against an agreed shape.

Confirmations

• I searched existing issues and this is not a duplicate.
• My reproduction uses shareable code (a dummy snippet or a public OSS repository), not proprietary code.

[kriswill]

Companion issues in this cascade:

#	Issue	Role in the cascade
#895	Fast artifact export raw-copies a live WAL-mode DB	Producer — torn graph.db.zst, weak import gate (this issue)
#896	Row-scan loops swallow SQLITE_CORRUPT	Concealer — total=N, results=[] with no error; let this artifact serve wrong answers
#897	Fresh DB install leaves old -wal/-shm behind	Second producer — stale WAL replayed over the new file
#898	Sequential pipeline emits invalid JSON	Gate blocker — aborts a quick_check-based import gate, so it interacts with the fix here