feat(stdlib): Sqlite.affine codegen foundation — db-theory #1a (6 externs wired Deno-ESM end-to-end)#522
Merged
Conversation
…erns wired Deno-ESM end-to-end)
First step of the database-theory functionality programme (see survey
in 2026-06-01 session). The audit found the SQLite surface had a
deceptive "partial" status: `stdlib/Sqlite.affine` declared 6 externs
(`db_open` / `db_close` / `db_execute` / `db_query` / `db_query_one` /
`db_query_int`) but NO codegen lowering existed in any backend, NO JS
runtime adapter shipped in the prelude, NO interpreter builtins were
registered, and ZERO consumer code or tests exercised the surface.
Functionally `Sqlite.affine` was vapourware behind a stable-looking
extern signature.
This PR lands the foundation so future extension PRs (prepared
statements, schema introspection, bulk I/O — db-theory #1b) have a
working pathway to build on:
## Codegen lowerings (`lib/codegen_deno.ml`)
6 new entries in `deno_builtins`, mapping each AffineScript extern to
the matching `__as_db*` JS helper.
## JS runtime helpers (`lib/codegen_deno.ml` prelude)
A new `// ---- Sqlite ----` block adds six `const __as_db*` helpers
that delegate to `globalThis.__as_sqlite`, an adapter namespace the
host installs once at startup. The shape mirrors the Motion / Pixi /
PixiSound binding pattern (the canonical "host installs a globalThis
shim" idiom in this repo) — Sqlite was the only Tier-2 binding without
one. Two concrete adapter recipes are inlined in the runtime comment:
- Deno: `jsr:@db/sqlite`'s `Database` class adapted to the contract.
- Node: `better-sqlite3`'s `Database` adapted likewise.
Parameter marshalling stays JSON-string-encoded to match the existing
extern signatures (richer typed bindings ship in #1b alongside a
prepared-statement `Stmt` type).
## Smoke harness (`tests/codegen-deno/sqlite_smoke.{affine,harness.mjs}`)
Three smoke functions exercise the full lifecycle, single-row binding
+ projection, and ordered multi-row retrieval. The `.harness.mjs`
installs a deliberately minimal in-memory SQL mock satisfying the
`__as_sqlite` contract — just enough CREATE / INSERT / SELECT / WHERE-?
/ ORDER BY / COUNT(*) subset to drive the 6 externs. The mock is
explicitly NOT a real SQL engine and the harness commentary directs
production deployments to swap to `@db/sqlite` or `better-sqlite3`.
## Verification
- `dune build bin/main.exe`: clean
- `dune runtest`: 357 / 357 green (codegen ⊆ stdlib consistency test
picks up the 6 new builtins and verifies the matching `pub extern fn`
decls exist in `stdlib/Sqlite.affine` — they do, from before)
- `tools/run_codegen_deno_tests.sh`: all 31 harnesses pass (was 30)
## db-theory #1b scope (next PR, separate)
- `Stmt` opaque type + `db_prepare` / `db_bind_int` / `db_bind_text` /
`db_bind_null` / `db_step` / `db_column_int` / `db_column_text` /
`db_finalize` for prepared statements.
- `db_schema_tables() -> [String]` + `db_schema_columns(table)` for
introspection.
- `db_import_csv` / `db_export_csv` for bulk I/O.
- Typed `DbError` extern + `Result<T, DbError>` returns where the JSON
string form is too lossy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🔍 Hypatia Security ScanFindings: 89 issues detected
View findings[
{
"reason": "Action perpolymath/standards/.github/workflows/governance-reusable.yml@main\n needs attention",
"type": "unpinned_action",
"file": "governance.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action ons/checkout@v6\n needs attention",
"type": "unpinned_action",
"file": "publish-jsr.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action land/setup-deno@v2\n needs attention",
"type": "unpinned_action",
"file": "publish-jsr.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in affine-vscode-publish.yml",
"type": "missing_timeout_minutes",
"file": "affine-vscode-publish.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in casket-pages.yml",
"type": "missing_timeout_minutes",
"file": "casket-pages.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in casket-pages.yml",
"type": "missing_timeout_minutes",
"file": "casket-pages.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in casket-pages.yml",
"type": "missing_timeout_minutes",
"file": "casket-pages.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in ci.yml",
"type": "missing_timeout_minutes",
"file": "ci.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in ci.yml",
"type": "missing_timeout_minutes",
"file": "ci.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in ci.yml",
"type": "missing_timeout_minutes",
"file": "ci.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
4 tasks
hyperpolymath
added a commit
that referenced
this pull request
Jun 1, 2026
…yped externs) (#524) ## Summary Layers the prepared-statement surface on top of the #522 convenience surface. Use this for anything carrying **user input**, anything iterating over **more than a handful of rows**, and anything where **typed value marshalling** matters (Int / Text / NULL). ## New stdlib surface (`stdlib/Sqlite.affine`) | extern | purpose | |---|---| | `extern type Stmt` | opaque statement handle | | `db_prepare(d, sql) -> Stmt` | compile SQL with `?` placeholders | | `db_bind_int(s, idx, v) -> Int` | bind 1-indexed param (sqlite3 convention) | | `db_bind_text(s, idx, v) -> Int` | bind text | | `db_bind_null(s, idx) -> Int` | bind NULL | | `db_step(s) -> Int` | `1` = `SQLITE_ROW`, `0` = `SQLITE_DONE` | | `db_column_count(s) -> Int` | current-row width | | `db_column_int(s, idx) -> Int` | 0-indexed read; NULL → 0 | | `db_column_text(s, idx) -> String` | 0-indexed read; NULL → "" | | `db_reset(s) -> Int` | re-step from row 0 without recompiling | | `db_finalize(s) -> Int` | release | Bind-index is 1-based, column-index is 0-based — matches both `jsr:@db/sqlite` and `better-sqlite3` (the two reference `__as_sqlite` adapters). NULL coercion happens at the JS-helper layer; callers needing NULL/value-zero discrimination use the convenience `db_query_one` path. ## Smoke harness (`tests/codegen-deno/sqlite_prepared.{affine,harness.mjs}`) | smoke fn | exercises | |---|---| | `smoke_prepare_bind_int_step_finalize` | `INSERT(?, ?)` with two int binds + finalize + query-back | | `smoke_step_iteration` | `SELECT` N rows, loop `db_step` until 0, accumulate via `db_column_int` | | `smoke_text_bind_and_column` | bind_text → step → column_text round-trip | | `smoke_null_bind` | bind_null → column_int coerces to 0 | | `smoke_reset_and_reuse` | prepare once + bind/step + reset + re-bind/step → 2 rows | | `smoke_column_count_basic` | `SELECT a, b, c FROM t` → column_count = 3 | Mock extends #522's in-memory adapter with 10 statement-side methods; production adapters (`jsr:@db/sqlite` / `better-sqlite3`) provide near-1:1 wrappers. ## Verification - `dune build bin/main.exe`: clean - `dune runtest`: **363 / 363** green (codegen ⊆ stdlib consistency picks up the 10 new builtins; all match `stdlib/Sqlite.affine`) - `tools/run_codegen_deno_tests.sh`: **32 / 32** harnesses pass (was 31) ## Stacked on #522 #522 merged 12:43Z; this branch forks from the resulting main HEAD. Clean diff. ## db-theory #1c (next PR, separate) - Schema introspection (`db_schema_tables`, `db_schema_columns`) - Bulk I/O (`db_import_csv`, `db_export_csv`) - Typed `DbError` + `Result<T, DbError>` variants ## Test plan - [x] dune build clean - [x] dune runtest 363/363 green - [x] tools/run_codegen_deno_tests.sh 32/32 pass - [x] 6 prepared-statement smoke assertions pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
hyperpolymath
added a commit
that referenced
this pull request
Jun 2, 2026
…on — db-theory #1c (6 externs) (#525) ## Summary Closes out the SQLite stdlib triad (#522 foundation + #524 prepared statements + this PR). Adds the three remaining practical surfaces: | layer | externs | |---|---| | **Schema introspection** | `db_schema_tables(d)`, `db_schema_columns(d, table)`, `db_table_exists(d, table)` | | **Bulk CSV I/O** | `db_import_csv(d, table, path, has_header)`, `db_export_csv(d, sql, params, path)` | | **Error inspection** | `db_last_error(d)` (typed `DbError` carrier lands in #1d) | ## Codegen + adapter 6 `b "name"` registrations + 6 `__as_db*` JS helpers, each delegating to one new method on `globalThis.__as_sqlite` (`schemaTables`, `schemaColumns`, `tableExists`, `importCsv`, `exportCsv`, `lastError`). Real adapters back these with one-line wrappers (PRAGMA table_info, prepare().iterate(), fs writes). ## Smoke harness (`tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs}`) 8 smoke functions × 9 assertions — schema_tables ordering, column descriptor shape (incl. PK flag), table_exists true/false, CSV import skipping header, CSV export including header, last_error empty/fault-injected. Mock extends the #1a/#1b adapter with the 6 new methods plus a virtual filesystem map for hermetic CSV testing. ## Verification - `dune build`: clean - `dune runtest`: **363 / 363** green - `tools/run_codegen_deno_tests.sh`: **33 / 33** harnesses pass (was 32) ## Stacked on #524 #524 (db-theory #1b — prepared statements) **MERGED 13:15Z**. This branch forks from its head and rebases cleanly. ## db-theory #1d (next PR, separate) - `extern type DbError` opaque carrier - `Result<T, DbError>`-shaped variants of the convenience surface - Design-blocked on the wider error-model question (do we add `Result` returns to all extern fns or keep the JSON-string sentinel pattern?) ## Test plan - [x] dune build clean - [x] dune runtest 363/363 green - [x] 33/33 codegen-deno harnesses pass - [x] 9 introspection/bulk/error smoke assertions pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 tasks
hyperpolymath
added a commit
that referenced
this pull request
Jun 2, 2026
## Summary Refresh the authoritative status surfaces in this repo to reflect the stdlib **db-theory triplet** that landed yesterday and today: - PR #525 — `Sqlite.affine` schema introspection + bulk I/O + error inspection (db-theory #1c, 6 externs) - PR #526 — `Transaction.affine` (db-theory #2, affine-bounded write-set isolation, 8 externs) - PR #527 — `Aggregate.affine` (db-theory #3, SQL group-by + aggregation primitives, 7 externs) - PR #528 — CHANGELOG sync (already merged) `CHANGELOG.md` is already up to date via #528, so this PR touches only the four files that ledger that work as status, not as history. ## What changed - `docs/CAPABILITY-MATRIX.adoc` — replace the stale "19/19 stdlib files" sentence (the AOT gate now discovers files dynamically) and call out the db-theory triplet as the current authoring frontier alongside Http / Json / Dict-Map. - `docs/TECH-DEBT.adoc` §C STDLIB — add a `STDLIB-05` row capturing the full Sqlite arc (#522 / #524 / #525), Transaction (#526), and Aggregate (#527), with pointers to the two academic proofs the PRs already shipped (`docs/academic/proofs/db-theory-{2,3}-*.md`). - `docs/stdlib-roadmap.adoc` — promote `Sqlite.affine` from `partial / Coverage audit needed` to `usable`, and add `Aggregate.affine` + `Transaction.affine` rows to the inventory snapshot. - `.machine_readable/6a2/STATE.a2ml` — bump `last-updated` to `2026-06-02` (this file mirrors, it does not lead — per its own drift-flag). ## What was deliberately *not* touched - `docs/ROADMAP.adoc` — language/compiler progress; the db-theory PRs are stdlib-side and don't move the language roadmap. - A `TEST-NEEDS` file — none exists at repo root; the three PRs each shipped their own Deno-ESM smoke harness under `tests/codegen-deno/` (`sqlite_introspect_bulk` / `transaction_smoke` / `aggregate_smoke`) and the AOT gate auto-picked the new modules up. - License / SPDX headers — per the estate's no-automated-licence-edits rule. ## Test plan - [x] `bash tools/check-doc-truthing.sh` passes locally — banner / primacy / mirror invariants intact, over-claim ratchet unchanged (no new baseline entry needed). - [x] Commit GPG-signed; verified `gpg: Good signature`. Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First step of the db-theory programme. SQLite's surface looked partial in the roadmap but was actually vapourware: 6 externs declared in
stdlib/Sqlite.affine, zero codegen / zero adapter / zero tests / zero consumers. This PR lands the foundation so the extension PR (#1b — prepared statements, schema introspection, bulk I/O) has a real pathway.Changes
lib/codegen_deno.mldeno_builtins— 6 new entries lowering each extern to its__as_db*JS helper.lib/codegen_deno.mlprelude —// ---- Sqlite ----block with 6const __as_db*helpers delegating toglobalThis.__as_sqlite. Same shape as Motion / Pixi / PixiSound (only Tier-2 binding without it). Inline comment gives the Deno (jsr:@db/sqlite) and Node (better-sqlite3) adapter recipes.tests/codegen-deno/sqlite_smoke.{affine,harness.mjs}— 3 smoke functions exercising full lifecycle,WHERE id = ?param-bound projection, and ordered multi-row retrieval. Harness installs an in-memory SQL mock (CREATE / INSERT / SELECT / WHERE / ORDER BY / COUNT(*) subset) explicitly marked as smoke-only.Verification
dune build bin/main.exe: cleandune runtest: 357 / 357 greentools/run_codegen_deno_tests.sh: all 31 harnesses pass (was 30)Next (db-theory #1b — separate PR)
Stmtopaque type +db_prepare/db_bind_*/db_step/db_column_*/db_finalizedb_schema_tables()+db_schema_columns(table)introspectiondb_import_csv/db_export_csvbulk I/ODbError+Result<T, DbError>returnsTest plan
dune buildcleandune runtest357 / 357 green (codegen ⊆ stdlib consistency picks up the 6 new builtins)tools/run_codegen_deno_tests.sh31 / 31 harnesses passsqlite_smokemock validates all three smoke functions🤖 Generated with Claude Code