feat(stdlib): Sqlite prepared statements — db-theory #1b (Stmt + 10 typed externs)#524
Merged
Merged
Conversation
…yped externs)
Layers the prepared-statement surface on top of the #1a convenience
surface. Use this for anything carrying user input, anything iterating
over more than a handful of rows, and anything where typed value
marshalling matters (Int / Text / NULL).
## New extern surface (`stdlib/Sqlite.affine`)
- `pub extern type Stmt` — opaque statement handle.
- `db_prepare(d, sql) -> Stmt` — compile SQL with `?` placeholders.
- `db_bind_int(s, idx, v) -> Int` — bind 1-indexed param (sqlite3 conv).
- `db_bind_text(s, idx, v) -> Int`
- `db_bind_null(s, idx) -> Int`
- `db_step(s) -> Int` — 1 if a row is available (`SQLITE_ROW`), 0 if
iteration is complete (`SQLITE_DONE`).
- `db_column_count(s) -> Int` — width of the current row.
- `db_column_int(s, idx) -> Int` — 0-indexed column read; NULL → 0.
- `db_column_text(s, idx) -> String` — 0-indexed column read; NULL → "".
- `db_reset(s) -> Int` — re-step from row 0 without recompiling.
- `db_finalize(s) -> Int` — release; handle invalid after.
Lifecycle + return-value conventions documented inline at the call
site. Bind-index is 1-based, column-index is 0-based — matches both
`jsr:@db/sqlite` and `better-sqlite3` (the two reference adapters
the `__as_sqlite` contract supports).
## Codegen lowerings (`lib/codegen_deno.ml`)
10 new entries in `deno_builtins`, each mapping the AffineScript extern
to a `__as_db*` JS helper. Each helper delegates to one
`globalThis.__as_sqlite.<method>` call, keeping the adapter contract
flat — the host adapter's `prepare`/`bindInt`/`bindText`/`bindNull`/
`step`/`columnCount`/`columnInt`/`columnText`/`reset`/`finalize`
methods are direct one-line passes through to the underlying sqlite
library.
NULL semantics handled at the JS-helper layer: `columnInt` / `columnText`
on a NULL column return `null` from the adapter, then coerce to `0` /
`""` respectively in the helper — matches the AffineScript-side type
signature (`Int` / `String`, no `Option`). Callers that need to
distinguish NULL from 0 / "" use the convenience `db_query_one` path
and inspect the JSON null in the result.
## Smoke harness (`tests/codegen-deno/sqlite_prepared.{affine,harness.mjs}`)
Six new smoke functions exercise the full prepared-statement lifecycle:
| smoke fn | exercises |
|---|---|
| `smoke_prepare_bind_int_step_finalize` | INSERT(?, ?) with two int binds + finalize + query-back |
| `smoke_step_iteration` | SELECT N rows, loop `db_step` until 0, accumulate `db_column_int` |
| `smoke_text_bind_and_column` | bind_text → step → column_text round-trip |
| `smoke_null_bind` | bind_null → column_int coerces to 0 |
| `smoke_reset_and_reuse` | prepare once + bind/step + reset + bind/step → 2 rows |
| `smoke_column_count_basic` | SELECT a, b, c → column_count = 3 |
The harness extends the #1a in-memory mock with the 10 statement-side
methods. Mock SQL parser stays narrow (CREATE / INSERT-with-binds /
SELECT / WHERE / ORDER BY / COUNT(*) / single-expression `SELECT a +
b`); harness commentary directs production deployments to the real
`__as_sqlite` adapter.
## Verification
- `dune build bin/main.exe`: clean
- `dune runtest`: **363 / 363** green (codegen ⊆ stdlib consistency
picks up the 10 new builtins; all match `stdlib/Sqlite.affine`)
- `tools/run_codegen_deno_tests.sh`: **all 32** harnesses pass (was 31)
## Stacked on #522 (db-theory #1a)
#522 merged 12:43Z; this branch forks from the resulting main HEAD.
## db-theory #1c scope (next PR, separate)
- Schema introspection: `db_schema_tables() -> [String]`,
`db_schema_columns(table: String) -> [String]`
- Bulk I/O: `db_import_csv(path) -> Int`, `db_export_csv(path) -> Int`
- Typed error: `extern type DbError` + `Result<T, DbError>`-shaped
variants of the convenience surface
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🔍 Hypatia Security ScanFindings: 88 issues detected
View findings[
{
"reason": "Action perpolymath/standards/.github/workflows/governance-reusable.yml@main\n needs attention",
"type": "unpinned_action",
"file": "governance.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action ons/checkout@v6\n needs attention",
"type": "unpinned_action",
"file": "publish-jsr.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action land/setup-deno@v2\n needs attention",
"type": "unpinned_action",
"file": "publish-jsr.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in affine-vscode-publish.yml",
"type": "missing_timeout_minutes",
"file": "affine-vscode-publish.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in casket-pages.yml",
"type": "missing_timeout_minutes",
"file": "casket-pages.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in casket-pages.yml",
"type": "missing_timeout_minutes",
"file": "casket-pages.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in casket-pages.yml",
"type": "missing_timeout_minutes",
"file": "casket-pages.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in ci.yml",
"type": "missing_timeout_minutes",
"file": "ci.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in ci.yml",
"type": "missing_timeout_minutes",
"file": "ci.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in ci.yml",
"type": "missing_timeout_minutes",
"file": "ci.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
4 tasks
hyperpolymath
added a commit
that referenced
this pull request
Jun 1, 2026
…on — db-theory #1c (6 externs)
Closes out the SQLite stdlib triad (#1a foundation + #1b prepared
statements + this PR). Adds the three remaining practical surfaces:
schema reflection, bulk CSV import/export, and a string-form
last-error inspector for the convenience path.
## New stdlib surface (`stdlib/Sqlite.affine`)
### Schema introspection (3 externs)
- `db_schema_tables(d) -> String` — JSON array of table names
(excludes `sqlite_*` internal namespace).
- `db_schema_columns(d, table) -> String` — JSON array of column
descriptors `{name, type, notnull, pk}`; `"[]"` if table absent.
- `db_table_exists(d, table) -> Bool` — convenience boolean over the
tables list.
### Bulk I/O (2 externs)
- `db_import_csv(d, table, csv_path, has_header) -> Int` — assumes the
destination table exists; returns rows inserted.
- `db_export_csv(d, sql, params_json, csv_path) -> Int` — runs `sql`
with `params_json`, writes one row per result-row plus a header row;
returns rows written.
### Error inspection (1 extern)
- `db_last_error(d) -> String` — `""` if no error has been recorded on
this connection. Disambiguates sentinels (e.g. `db_query_one`
returning `"null"`) from actual failures. A typed `DbError` carrier
+ `Result<T, DbError>`-shaped variants land in db-theory #1d
(separate, design-blocked on the wider error-model question).
## Codegen lowerings (`lib/codegen_deno.ml`)
6 new `b "name"` registrations + 6 corresponding `__as_db*` JS
helpers. Each helper is one delegated call to a new method on the
`globalThis.__as_sqlite` adapter (`schemaTables`, `schemaColumns`,
`tableExists`, `importCsv`, `exportCsv`, `lastError`). Real adapters
back these with one-line wrappers: `PRAGMA table_info` for schema,
`Database.prepare().iterate()` + a CSV serialiser for export, etc.
## Smoke harness (`tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs}`)
8 smoke functions × 9 assertions cover:
- `db_schema_tables` returns the two created tables in lexicographic
order, with `sqlite_*`-prefixed names excluded.
- `db_schema_columns` returns 3 column descriptors; first column
flagged as PK (mock heuristic mirrors sqlite3's `INTEGER PRIMARY KEY`).
- `db_table_exists` true/false branches.
- `db_import_csv` skips the header row and inserts 3 data rows from
an in-memory CSV.
- `db_export_csv` writes header + data rows to a virtual filesystem
path; assertions inspect the written content.
- `db_last_error` returns `""` on a clean connection and the
fault-injected message after a `RAISE 'simulated failure'` execute.
Mock extends the #1a/#1b in-memory adapter with the 6 new methods
plus a virtual filesystem map (path → string contents) so the CSV
paths exercise no real disk I/O — same isolation discipline as the
existing harnesses.
## Verification
- `dune build bin/main.exe`: clean
- `dune runtest`: **363 / 363** green (codegen ⊆ stdlib consistency
picks up the 6 new builtins; all match `stdlib/Sqlite.affine`)
- `tools/run_codegen_deno_tests.sh`: **all 33** harnesses pass (was 32)
## Stacked on #524 (db-theory #1b)
#524 finishes the prepared-statement surface; this branch forks from
its head. Both rebase cleanly onto main as they merge.
## db-theory #1d scope (next PR, separate)
- `extern type DbError` opaque carrier + accessor externs.
- `Result<T, DbError>`-shaped variants of `db_open`, `db_execute`,
`db_query`, `db_query_one`, `db_query_int` (the convenience surface
where the JSON-string error path is too lossy).
- Design note: prepared-statement surface already raises through the
AffineScript-side `try`/`catch` desugar, so its `Result`-isation is
not in scope unless concrete consumers ask.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hyperpolymath
added a commit
that referenced
this pull request
Jun 2, 2026
…on — db-theory #1c (6 externs) (#525) ## Summary Closes out the SQLite stdlib triad (#522 foundation + #524 prepared statements + this PR). Adds the three remaining practical surfaces: | layer | externs | |---|---| | **Schema introspection** | `db_schema_tables(d)`, `db_schema_columns(d, table)`, `db_table_exists(d, table)` | | **Bulk CSV I/O** | `db_import_csv(d, table, path, has_header)`, `db_export_csv(d, sql, params, path)` | | **Error inspection** | `db_last_error(d)` (typed `DbError` carrier lands in #1d) | ## Codegen + adapter 6 `b "name"` registrations + 6 `__as_db*` JS helpers, each delegating to one new method on `globalThis.__as_sqlite` (`schemaTables`, `schemaColumns`, `tableExists`, `importCsv`, `exportCsv`, `lastError`). Real adapters back these with one-line wrappers (PRAGMA table_info, prepare().iterate(), fs writes). ## Smoke harness (`tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs}`) 8 smoke functions × 9 assertions — schema_tables ordering, column descriptor shape (incl. PK flag), table_exists true/false, CSV import skipping header, CSV export including header, last_error empty/fault-injected. Mock extends the #1a/#1b adapter with the 6 new methods plus a virtual filesystem map for hermetic CSV testing. ## Verification - `dune build`: clean - `dune runtest`: **363 / 363** green - `tools/run_codegen_deno_tests.sh`: **33 / 33** harnesses pass (was 32) ## Stacked on #524 #524 (db-theory #1b — prepared statements) **MERGED 13:15Z**. This branch forks from its head and rebases cleanly. ## db-theory #1d (next PR, separate) - `extern type DbError` opaque carrier - `Result<T, DbError>`-shaped variants of the convenience surface - Design-blocked on the wider error-model question (do we add `Result` returns to all extern fns or keep the JSON-string sentinel pattern?) ## Test plan - [x] dune build clean - [x] dune runtest 363/363 green - [x] 33/33 codegen-deno harnesses pass - [x] 9 introspection/bulk/error smoke assertions pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 tasks
hyperpolymath
added a commit
that referenced
this pull request
Jun 2, 2026
## Summary Refresh the authoritative status surfaces in this repo to reflect the stdlib **db-theory triplet** that landed yesterday and today: - PR #525 — `Sqlite.affine` schema introspection + bulk I/O + error inspection (db-theory #1c, 6 externs) - PR #526 — `Transaction.affine` (db-theory #2, affine-bounded write-set isolation, 8 externs) - PR #527 — `Aggregate.affine` (db-theory #3, SQL group-by + aggregation primitives, 7 externs) - PR #528 — CHANGELOG sync (already merged) `CHANGELOG.md` is already up to date via #528, so this PR touches only the four files that ledger that work as status, not as history. ## What changed - `docs/CAPABILITY-MATRIX.adoc` — replace the stale "19/19 stdlib files" sentence (the AOT gate now discovers files dynamically) and call out the db-theory triplet as the current authoring frontier alongside Http / Json / Dict-Map. - `docs/TECH-DEBT.adoc` §C STDLIB — add a `STDLIB-05` row capturing the full Sqlite arc (#522 / #524 / #525), Transaction (#526), and Aggregate (#527), with pointers to the two academic proofs the PRs already shipped (`docs/academic/proofs/db-theory-{2,3}-*.md`). - `docs/stdlib-roadmap.adoc` — promote `Sqlite.affine` from `partial / Coverage audit needed` to `usable`, and add `Aggregate.affine` + `Transaction.affine` rows to the inventory snapshot. - `.machine_readable/6a2/STATE.a2ml` — bump `last-updated` to `2026-06-02` (this file mirrors, it does not lead — per its own drift-flag). ## What was deliberately *not* touched - `docs/ROADMAP.adoc` — language/compiler progress; the db-theory PRs are stdlib-side and don't move the language roadmap. - A `TEST-NEEDS` file — none exists at repo root; the three PRs each shipped their own Deno-ESM smoke harness under `tests/codegen-deno/` (`sqlite_introspect_bulk` / `transaction_smoke` / `aggregate_smoke`) and the AOT gate auto-picked the new modules up. - License / SPDX headers — per the estate's no-automated-licence-edits rule. ## Test plan - [x] `bash tools/check-doc-truthing.sh` passes locally — banner / primacy / mirror invariants intact, over-claim ratchet unchanged (no new baseline entry needed). - [x] Commit GPG-signed; verified `gpg: Good signature`. Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Layers the prepared-statement surface on top of the #522 convenience surface. Use this for anything carrying user input, anything iterating over more than a handful of rows, and anything where typed value marshalling matters (Int / Text / NULL).
New stdlib surface (
stdlib/Sqlite.affine)extern type Stmtdb_prepare(d, sql) -> Stmt?placeholdersdb_bind_int(s, idx, v) -> Intdb_bind_text(s, idx, v) -> Intdb_bind_null(s, idx) -> Intdb_step(s) -> Int1=SQLITE_ROW,0=SQLITE_DONEdb_column_count(s) -> Intdb_column_int(s, idx) -> Intdb_column_text(s, idx) -> Stringdb_reset(s) -> Intdb_finalize(s) -> IntBind-index is 1-based, column-index is 0-based — matches both
jsr:@db/sqliteandbetter-sqlite3(the two reference__as_sqliteadapters). NULL coercion happens at the JS-helper layer; callers needing NULL/value-zero discrimination use the conveniencedb_query_onepath.Smoke harness (
tests/codegen-deno/sqlite_prepared.{affine,harness.mjs})smoke_prepare_bind_int_step_finalizeINSERT(?, ?)with two int binds + finalize + query-backsmoke_step_iterationSELECTN rows, loopdb_stepuntil 0, accumulate viadb_column_intsmoke_text_bind_and_columnsmoke_null_bindsmoke_reset_and_reusesmoke_column_count_basicSELECT a, b, c FROM t→ column_count = 3Mock extends #522's in-memory adapter with 10 statement-side methods; production adapters (
jsr:@db/sqlite/better-sqlite3) provide near-1:1 wrappers.Verification
dune build bin/main.exe: cleandune runtest: 363 / 363 green (codegen ⊆ stdlib consistency picks up the 10 new builtins; all matchstdlib/Sqlite.affine)tools/run_codegen_deno_tests.sh: 32 / 32 harnesses pass (was 31)Stacked on #522
#522 merged 12:43Z; this branch forks from the resulting main HEAD. Clean diff.
db-theory #1c (next PR, separate)
db_schema_tables,db_schema_columns)db_import_csv,db_export_csv)DbError+Result<T, DbError>variantsTest plan
🤖 Generated with Claude Code