Skip to content

feat(stdlib): Sqlite prepared statements — db-theory #1b (Stmt + 10 typed externs)#524

Merged
hyperpolymath merged 1 commit into
mainfrom
feat/db-1b-sqlite-prepared
Jun 1, 2026
Merged

feat(stdlib): Sqlite prepared statements — db-theory #1b (Stmt + 10 typed externs)#524
hyperpolymath merged 1 commit into
mainfrom
feat/db-1b-sqlite-prepared

Conversation

@hyperpolymath
Copy link
Copy Markdown
Owner

Summary

Layers the prepared-statement surface on top of the #522 convenience surface. Use this for anything carrying user input, anything iterating over more than a handful of rows, and anything where typed value marshalling matters (Int / Text / NULL).

New stdlib surface (stdlib/Sqlite.affine)

extern purpose
extern type Stmt opaque statement handle
db_prepare(d, sql) -> Stmt compile SQL with ? placeholders
db_bind_int(s, idx, v) -> Int bind 1-indexed param (sqlite3 convention)
db_bind_text(s, idx, v) -> Int bind text
db_bind_null(s, idx) -> Int bind NULL
db_step(s) -> Int 1 = SQLITE_ROW, 0 = SQLITE_DONE
db_column_count(s) -> Int current-row width
db_column_int(s, idx) -> Int 0-indexed read; NULL → 0
db_column_text(s, idx) -> String 0-indexed read; NULL → ""
db_reset(s) -> Int re-step from row 0 without recompiling
db_finalize(s) -> Int release

Bind-index is 1-based, column-index is 0-based — matches both jsr:@db/sqlite and better-sqlite3 (the two reference __as_sqlite adapters). NULL coercion happens at the JS-helper layer; callers needing NULL/value-zero discrimination use the convenience db_query_one path.

Smoke harness (tests/codegen-deno/sqlite_prepared.{affine,harness.mjs})

smoke fn exercises
smoke_prepare_bind_int_step_finalize INSERT(?, ?) with two int binds + finalize + query-back
smoke_step_iteration SELECT N rows, loop db_step until 0, accumulate via db_column_int
smoke_text_bind_and_column bind_text → step → column_text round-trip
smoke_null_bind bind_null → column_int coerces to 0
smoke_reset_and_reuse prepare once + bind/step + reset + re-bind/step → 2 rows
smoke_column_count_basic SELECT a, b, c FROM t → column_count = 3

Mock extends #522's in-memory adapter with 10 statement-side methods; production adapters (jsr:@db/sqlite / better-sqlite3) provide near-1:1 wrappers.

Verification

  • dune build bin/main.exe: clean
  • dune runtest: 363 / 363 green (codegen ⊆ stdlib consistency picks up the 10 new builtins; all match stdlib/Sqlite.affine)
  • tools/run_codegen_deno_tests.sh: 32 / 32 harnesses pass (was 31)

Stacked on #522

#522 merged 12:43Z; this branch forks from the resulting main HEAD. Clean diff.

db-theory #1c (next PR, separate)

  • Schema introspection (db_schema_tables, db_schema_columns)
  • Bulk I/O (db_import_csv, db_export_csv)
  • Typed DbError + Result<T, DbError> variants

Test plan

  • dune build clean
  • dune runtest 363/363 green
  • tools/run_codegen_deno_tests.sh 32/32 pass
  • 6 prepared-statement smoke assertions pass

🤖 Generated with Claude Code

…yped externs)

Layers the prepared-statement surface on top of the #1a convenience
surface. Use this for anything carrying user input, anything iterating
over more than a handful of rows, and anything where typed value
marshalling matters (Int / Text / NULL).

## New extern surface (`stdlib/Sqlite.affine`)

- `pub extern type Stmt` — opaque statement handle.
- `db_prepare(d, sql) -> Stmt` — compile SQL with `?` placeholders.
- `db_bind_int(s, idx, v) -> Int` — bind 1-indexed param (sqlite3 conv).
- `db_bind_text(s, idx, v) -> Int`
- `db_bind_null(s, idx) -> Int`
- `db_step(s) -> Int` — 1 if a row is available (`SQLITE_ROW`), 0 if
  iteration is complete (`SQLITE_DONE`).
- `db_column_count(s) -> Int` — width of the current row.
- `db_column_int(s, idx) -> Int` — 0-indexed column read; NULL → 0.
- `db_column_text(s, idx) -> String` — 0-indexed column read; NULL → "".
- `db_reset(s) -> Int` — re-step from row 0 without recompiling.
- `db_finalize(s) -> Int` — release; handle invalid after.

Lifecycle + return-value conventions documented inline at the call
site. Bind-index is 1-based, column-index is 0-based — matches both
`jsr:@db/sqlite` and `better-sqlite3` (the two reference adapters
the `__as_sqlite` contract supports).

## Codegen lowerings (`lib/codegen_deno.ml`)

10 new entries in `deno_builtins`, each mapping the AffineScript extern
to a `__as_db*` JS helper. Each helper delegates to one
`globalThis.__as_sqlite.<method>` call, keeping the adapter contract
flat — the host adapter's `prepare`/`bindInt`/`bindText`/`bindNull`/
`step`/`columnCount`/`columnInt`/`columnText`/`reset`/`finalize`
methods are direct one-line passes through to the underlying sqlite
library.

NULL semantics handled at the JS-helper layer: `columnInt` / `columnText`
on a NULL column return `null` from the adapter, then coerce to `0` /
`""` respectively in the helper — matches the AffineScript-side type
signature (`Int` / `String`, no `Option`). Callers that need to
distinguish NULL from 0 / "" use the convenience `db_query_one` path
and inspect the JSON null in the result.

## Smoke harness (`tests/codegen-deno/sqlite_prepared.{affine,harness.mjs}`)

Six new smoke functions exercise the full prepared-statement lifecycle:

| smoke fn | exercises |
|---|---|
| `smoke_prepare_bind_int_step_finalize` | INSERT(?, ?) with two int binds + finalize + query-back |
| `smoke_step_iteration` | SELECT N rows, loop `db_step` until 0, accumulate `db_column_int` |
| `smoke_text_bind_and_column` | bind_text → step → column_text round-trip |
| `smoke_null_bind` | bind_null → column_int coerces to 0 |
| `smoke_reset_and_reuse` | prepare once + bind/step + reset + bind/step → 2 rows |
| `smoke_column_count_basic` | SELECT a, b, c → column_count = 3 |

The harness extends the #1a in-memory mock with the 10 statement-side
methods. Mock SQL parser stays narrow (CREATE / INSERT-with-binds /
SELECT / WHERE / ORDER BY / COUNT(*) / single-expression `SELECT a +
b`); harness commentary directs production deployments to the real
`__as_sqlite` adapter.

## Verification

- `dune build bin/main.exe`: clean
- `dune runtest`: **363 / 363** green (codegen ⊆ stdlib consistency
  picks up the 10 new builtins; all match `stdlib/Sqlite.affine`)
- `tools/run_codegen_deno_tests.sh`: **all 32** harnesses pass (was 31)

## Stacked on #522 (db-theory #1a)

#522 merged 12:43Z; this branch forks from the resulting main HEAD.

## db-theory #1c scope (next PR, separate)

- Schema introspection: `db_schema_tables() -> [String]`,
  `db_schema_columns(table: String) -> [String]`
- Bulk I/O: `db_import_csv(path) -> Int`, `db_export_csv(path) -> Int`
- Typed error: `extern type DbError` + `Result<T, DbError>`-shaped
  variants of the convenience surface

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@hyperpolymath hyperpolymath enabled auto-merge (squash) June 1, 2026 13:11
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 1, 2026

🔍 Hypatia Security Scan

Findings: 88 issues detected

Severity Count
🔴 Critical 2
🟠 High 16
🟡 Medium 70

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Action perpolymath/standards/.github/workflows/governance-reusable.yml@main\n needs attention",
    "type": "unpinned_action",
    "file": "governance.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action ons/checkout@v6\n    needs attention",
    "type": "unpinned_action",
    "file": "publish-jsr.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action land/setup-deno@v2\n    needs attention",
    "type": "unpinned_action",
    "file": "publish-jsr.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in affine-vscode-publish.yml",
    "type": "missing_timeout_minutes",
    "file": "affine-vscode-publish.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@hyperpolymath hyperpolymath merged commit 0f88028 into main Jun 1, 2026
27 checks passed
@hyperpolymath hyperpolymath deleted the feat/db-1b-sqlite-prepared branch June 1, 2026 13:15
hyperpolymath added a commit that referenced this pull request Jun 1, 2026
…on — db-theory #1c (6 externs)

Closes out the SQLite stdlib triad (#1a foundation + #1b prepared
statements + this PR). Adds the three remaining practical surfaces:
schema reflection, bulk CSV import/export, and a string-form
last-error inspector for the convenience path.

## New stdlib surface (`stdlib/Sqlite.affine`)

### Schema introspection (3 externs)
- `db_schema_tables(d) -> String` — JSON array of table names
  (excludes `sqlite_*` internal namespace).
- `db_schema_columns(d, table) -> String` — JSON array of column
  descriptors `{name, type, notnull, pk}`; `"[]"` if table absent.
- `db_table_exists(d, table) -> Bool` — convenience boolean over the
  tables list.

### Bulk I/O (2 externs)
- `db_import_csv(d, table, csv_path, has_header) -> Int` — assumes the
  destination table exists; returns rows inserted.
- `db_export_csv(d, sql, params_json, csv_path) -> Int` — runs `sql`
  with `params_json`, writes one row per result-row plus a header row;
  returns rows written.

### Error inspection (1 extern)
- `db_last_error(d) -> String` — `""` if no error has been recorded on
  this connection. Disambiguates sentinels (e.g. `db_query_one`
  returning `"null"`) from actual failures. A typed `DbError` carrier
  + `Result<T, DbError>`-shaped variants land in db-theory #1d
  (separate, design-blocked on the wider error-model question).

## Codegen lowerings (`lib/codegen_deno.ml`)

6 new `b "name"` registrations + 6 corresponding `__as_db*` JS
helpers. Each helper is one delegated call to a new method on the
`globalThis.__as_sqlite` adapter (`schemaTables`, `schemaColumns`,
`tableExists`, `importCsv`, `exportCsv`, `lastError`). Real adapters
back these with one-line wrappers: `PRAGMA table_info` for schema,
`Database.prepare().iterate()` + a CSV serialiser for export, etc.

## Smoke harness (`tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs}`)

8 smoke functions × 9 assertions cover:
- `db_schema_tables` returns the two created tables in lexicographic
  order, with `sqlite_*`-prefixed names excluded.
- `db_schema_columns` returns 3 column descriptors; first column
  flagged as PK (mock heuristic mirrors sqlite3's `INTEGER PRIMARY KEY`).
- `db_table_exists` true/false branches.
- `db_import_csv` skips the header row and inserts 3 data rows from
  an in-memory CSV.
- `db_export_csv` writes header + data rows to a virtual filesystem
  path; assertions inspect the written content.
- `db_last_error` returns `""` on a clean connection and the
  fault-injected message after a `RAISE 'simulated failure'` execute.

Mock extends the #1a/#1b in-memory adapter with the 6 new methods
plus a virtual filesystem map (path → string contents) so the CSV
paths exercise no real disk I/O — same isolation discipline as the
existing harnesses.

## Verification

- `dune build bin/main.exe`: clean
- `dune runtest`: **363 / 363** green (codegen ⊆ stdlib consistency
  picks up the 6 new builtins; all match `stdlib/Sqlite.affine`)
- `tools/run_codegen_deno_tests.sh`: **all 33** harnesses pass (was 32)

## Stacked on #524 (db-theory #1b)

#524 finishes the prepared-statement surface; this branch forks from
its head. Both rebase cleanly onto main as they merge.

## db-theory #1d scope (next PR, separate)

- `extern type DbError` opaque carrier + accessor externs.
- `Result<T, DbError>`-shaped variants of `db_open`, `db_execute`,
  `db_query`, `db_query_one`, `db_query_int` (the convenience surface
  where the JSON-string error path is too lossy).
- Design note: prepared-statement surface already raises through the
  AffineScript-side `try`/`catch` desugar, so its `Result`-isation is
  not in scope unless concrete consumers ask.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hyperpolymath added a commit that referenced this pull request Jun 2, 2026
…on — db-theory #1c (6 externs) (#525)

## Summary

Closes out the SQLite stdlib triad (#522 foundation + #524 prepared
statements + this PR). Adds the three remaining practical surfaces:

| layer | externs |
|---|---|
| **Schema introspection** | `db_schema_tables(d)`,
`db_schema_columns(d, table)`, `db_table_exists(d, table)` |
| **Bulk CSV I/O** | `db_import_csv(d, table, path, has_header)`,
`db_export_csv(d, sql, params, path)` |
| **Error inspection** | `db_last_error(d)` (typed `DbError` carrier
lands in #1d) |

## Codegen + adapter

6 `b "name"` registrations + 6 `__as_db*` JS helpers, each delegating to
one new method on `globalThis.__as_sqlite` (`schemaTables`,
`schemaColumns`, `tableExists`, `importCsv`, `exportCsv`, `lastError`).
Real adapters back these with one-line wrappers (PRAGMA table_info,
prepare().iterate(), fs writes).

## Smoke harness
(`tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs}`)

8 smoke functions × 9 assertions — schema_tables ordering, column
descriptor shape (incl. PK flag), table_exists true/false, CSV import
skipping header, CSV export including header, last_error
empty/fault-injected. Mock extends the #1a/#1b adapter with the 6 new
methods plus a virtual filesystem map for hermetic CSV testing.

## Verification

- `dune build`: clean
- `dune runtest`: **363 / 363** green
- `tools/run_codegen_deno_tests.sh`: **33 / 33** harnesses pass (was 32)

## Stacked on #524

#524 (db-theory #1b — prepared statements) **MERGED 13:15Z**. This
branch forks from its head and rebases cleanly.

## db-theory #1d (next PR, separate)

- `extern type DbError` opaque carrier
- `Result<T, DbError>`-shaped variants of the convenience surface
- Design-blocked on the wider error-model question (do we add `Result`
returns to all extern fns or keep the JSON-string sentinel pattern?)

## Test plan

- [x] dune build clean
- [x] dune runtest 363/363 green
- [x] 33/33 codegen-deno harnesses pass
- [x] 9 introspection/bulk/error smoke assertions pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hyperpolymath added a commit that referenced this pull request Jun 2, 2026
## Summary

Refresh the authoritative status surfaces in this repo to reflect the
stdlib **db-theory triplet** that landed yesterday and today:

- PR #525 — `Sqlite.affine` schema introspection + bulk I/O + error
inspection (db-theory #1c, 6 externs)
- PR #526 — `Transaction.affine` (db-theory #2, affine-bounded write-set
isolation, 8 externs)
- PR #527 — `Aggregate.affine` (db-theory #3, SQL group-by + aggregation
primitives, 7 externs)
- PR #528 — CHANGELOG sync (already merged)

`CHANGELOG.md` is already up to date via #528, so this PR touches only
the four files that ledger that work as status, not as history.

## What changed

- `docs/CAPABILITY-MATRIX.adoc` — replace the stale "19/19 stdlib files"
sentence (the AOT gate now discovers files dynamically) and call out the
db-theory triplet as the current authoring frontier alongside Http /
Json / Dict-Map.
- `docs/TECH-DEBT.adoc` §C STDLIB — add a `STDLIB-05` row capturing the
full Sqlite arc (#522 / #524 / #525), Transaction (#526), and Aggregate
(#527), with pointers to the two academic proofs the PRs already shipped
(`docs/academic/proofs/db-theory-{2,3}-*.md`).
- `docs/stdlib-roadmap.adoc` — promote `Sqlite.affine` from `partial /
Coverage audit needed` to `usable`, and add `Aggregate.affine` +
`Transaction.affine` rows to the inventory snapshot.
- `.machine_readable/6a2/STATE.a2ml` — bump `last-updated` to
`2026-06-02` (this file mirrors, it does not lead — per its own
drift-flag).

## What was deliberately *not* touched

- `docs/ROADMAP.adoc` — language/compiler progress; the db-theory PRs
are stdlib-side and don't move the language roadmap.
- A `TEST-NEEDS` file — none exists at repo root; the three PRs each
shipped their own Deno-ESM smoke harness under `tests/codegen-deno/`
(`sqlite_introspect_bulk` / `transaction_smoke` / `aggregate_smoke`) and
the AOT gate auto-picked the new modules up.
- License / SPDX headers — per the estate's no-automated-licence-edits
rule.

## Test plan

- [x] `bash tools/check-doc-truthing.sh` passes locally — banner /
primacy / mirror invariants intact, over-claim ratchet unchanged (no new
baseline entry needed).
- [x] Commit GPG-signed; verified `gpg: Good signature`.

Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant