Skip to content

feat(stdlib): Sqlite schema introspection + bulk I/O + error inspection — db-theory #1c (6 externs)#525

Merged
hyperpolymath merged 1 commit into
mainfrom
feat/db-1c-sqlite-introspect-bulk-error
Jun 2, 2026
Merged

feat(stdlib): Sqlite schema introspection + bulk I/O + error inspection — db-theory #1c (6 externs)#525
hyperpolymath merged 1 commit into
mainfrom
feat/db-1c-sqlite-introspect-bulk-error

Conversation

@hyperpolymath
Copy link
Copy Markdown
Owner

Summary

Closes out the SQLite stdlib triad (#522 foundation + #524 prepared statements + this PR). Adds the three remaining practical surfaces:

layer externs
Schema introspection db_schema_tables(d), db_schema_columns(d, table), db_table_exists(d, table)
Bulk CSV I/O db_import_csv(d, table, path, has_header), db_export_csv(d, sql, params, path)
Error inspection db_last_error(d) (typed DbError carrier lands in #1d)

Codegen + adapter

6 b "name" registrations + 6 __as_db* JS helpers, each delegating to one new method on globalThis.__as_sqlite (schemaTables, schemaColumns, tableExists, importCsv, exportCsv, lastError). Real adapters back these with one-line wrappers (PRAGMA table_info, prepare().iterate(), fs writes).

Smoke harness (tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs})

8 smoke functions × 9 assertions — schema_tables ordering, column descriptor shape (incl. PK flag), table_exists true/false, CSV import skipping header, CSV export including header, last_error empty/fault-injected. Mock extends the #1a/#1b adapter with the 6 new methods plus a virtual filesystem map for hermetic CSV testing.

Verification

  • dune build: clean
  • dune runtest: 363 / 363 green
  • tools/run_codegen_deno_tests.sh: 33 / 33 harnesses pass (was 32)

Stacked on #524

#524 (db-theory #1b — prepared statements) MERGED 13:15Z. This branch forks from its head and rebases cleanly.

db-theory #1d (next PR, separate)

  • extern type DbError opaque carrier
  • Result<T, DbError>-shaped variants of the convenience surface
  • Design-blocked on the wider error-model question (do we add Result returns to all extern fns or keep the JSON-string sentinel pattern?)

Test plan

  • dune build clean
  • dune runtest 363/363 green
  • 33/33 codegen-deno harnesses pass
  • 9 introspection/bulk/error smoke assertions pass

🤖 Generated with Claude Code

@hyperpolymath hyperpolymath enabled auto-merge (squash) June 1, 2026 13:16
…on — db-theory #1c (6 externs)

Closes out the SQLite stdlib triad (#1a foundation + #1b prepared
statements + this PR). Adds the three remaining practical surfaces:
schema reflection, bulk CSV import/export, and a string-form
last-error inspector for the convenience path.

## New stdlib surface (`stdlib/Sqlite.affine`)

### Schema introspection (3 externs)
- `db_schema_tables(d) -> String` — JSON array of table names
  (excludes `sqlite_*` internal namespace).
- `db_schema_columns(d, table) -> String` — JSON array of column
  descriptors `{name, type, notnull, pk}`; `"[]"` if table absent.
- `db_table_exists(d, table) -> Bool` — convenience boolean over the
  tables list.

### Bulk I/O (2 externs)
- `db_import_csv(d, table, csv_path, has_header) -> Int` — assumes the
  destination table exists; returns rows inserted.
- `db_export_csv(d, sql, params_json, csv_path) -> Int` — runs `sql`
  with `params_json`, writes one row per result-row plus a header row;
  returns rows written.

### Error inspection (1 extern)
- `db_last_error(d) -> String` — `""` if no error has been recorded on
  this connection. Disambiguates sentinels (e.g. `db_query_one`
  returning `"null"`) from actual failures. A typed `DbError` carrier
  + `Result<T, DbError>`-shaped variants land in db-theory #1d
  (separate, design-blocked on the wider error-model question).

## Codegen lowerings (`lib/codegen_deno.ml`)

6 new `b "name"` registrations + 6 corresponding `__as_db*` JS
helpers. Each helper is one delegated call to a new method on the
`globalThis.__as_sqlite` adapter (`schemaTables`, `schemaColumns`,
`tableExists`, `importCsv`, `exportCsv`, `lastError`). Real adapters
back these with one-line wrappers: `PRAGMA table_info` for schema,
`Database.prepare().iterate()` + a CSV serialiser for export, etc.

## Smoke harness (`tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs}`)

8 smoke functions × 9 assertions cover:
- `db_schema_tables` returns the two created tables in lexicographic
  order, with `sqlite_*`-prefixed names excluded.
- `db_schema_columns` returns 3 column descriptors; first column
  flagged as PK (mock heuristic mirrors sqlite3's `INTEGER PRIMARY KEY`).
- `db_table_exists` true/false branches.
- `db_import_csv` skips the header row and inserts 3 data rows from
  an in-memory CSV.
- `db_export_csv` writes header + data rows to a virtual filesystem
  path; assertions inspect the written content.
- `db_last_error` returns `""` on a clean connection and the
  fault-injected message after a `RAISE 'simulated failure'` execute.

Mock extends the #1a/#1b in-memory adapter with the 6 new methods
plus a virtual filesystem map (path → string contents) so the CSV
paths exercise no real disk I/O — same isolation discipline as the
existing harnesses.

## Verification

- `dune build bin/main.exe`: clean
- `dune runtest`: **363 / 363** green (codegen ⊆ stdlib consistency
  picks up the 6 new builtins; all match `stdlib/Sqlite.affine`)
- `tools/run_codegen_deno_tests.sh`: **all 33** harnesses pass (was 32)

## Stacked on #524 (db-theory #1b)

#524 finishes the prepared-statement surface; this branch forks from
its head. Both rebase cleanly onto main as they merge.

## db-theory #1d scope (next PR, separate)

- `extern type DbError` opaque carrier + accessor externs.
- `Result<T, DbError>`-shaped variants of `db_open`, `db_execute`,
  `db_query`, `db_query_one`, `db_query_int` (the convenience surface
  where the JSON-string error path is too lossy).
- Design note: prepared-statement surface already raises through the
  AffineScript-side `try`/`catch` desugar, so its `Result`-isation is
  not in scope unless concrete consumers ask.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@hyperpolymath hyperpolymath force-pushed the feat/db-1c-sqlite-introspect-bulk-error branch from 393c344 to 42b5b54 Compare June 1, 2026 13:31
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 1, 2026

🔍 Hypatia Security Scan

Findings: 88 issues detected

Severity Count
🔴 Critical 2
🟠 High 16
🟡 Medium 70

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Action perpolymath/standards/.github/workflows/governance-reusable.yml@main\n needs attention",
    "type": "unpinned_action",
    "file": "governance.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action ons/checkout@v6\n    needs attention",
    "type": "unpinned_action",
    "file": "publish-jsr.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action land/setup-deno@v2\n    needs attention",
    "type": "unpinned_action",
    "file": "publish-jsr.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in affine-vscode-publish.yml",
    "type": "missing_timeout_minutes",
    "file": "affine-vscode-publish.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@hyperpolymath hyperpolymath merged commit d7cdec5 into main Jun 2, 2026
27 checks passed
@hyperpolymath hyperpolymath deleted the feat/db-1c-sqlite-introspect-bulk-error branch June 2, 2026 09:16
hyperpolymath added a commit that referenced this pull request Jun 2, 2026
Appends today's three db-theory stdlib PRs to the [Unreleased] section.
Repo's changelog-reusable.yml automation isn't wired yet (manual append
per the file's own header pointer).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hyperpolymath added a commit that referenced this pull request Jun 2, 2026
## Summary

Refresh the authoritative status surfaces in this repo to reflect the
stdlib **db-theory triplet** that landed yesterday and today:

- PR #525 — `Sqlite.affine` schema introspection + bulk I/O + error
inspection (db-theory #1c, 6 externs)
- PR #526 — `Transaction.affine` (db-theory #2, affine-bounded write-set
isolation, 8 externs)
- PR #527 — `Aggregate.affine` (db-theory #3, SQL group-by + aggregation
primitives, 7 externs)
- PR #528 — CHANGELOG sync (already merged)

`CHANGELOG.md` is already up to date via #528, so this PR touches only
the four files that ledger that work as status, not as history.

## What changed

- `docs/CAPABILITY-MATRIX.adoc` — replace the stale "19/19 stdlib files"
sentence (the AOT gate now discovers files dynamically) and call out the
db-theory triplet as the current authoring frontier alongside Http /
Json / Dict-Map.
- `docs/TECH-DEBT.adoc` §C STDLIB — add a `STDLIB-05` row capturing the
full Sqlite arc (#522 / #524 / #525), Transaction (#526), and Aggregate
(#527), with pointers to the two academic proofs the PRs already shipped
(`docs/academic/proofs/db-theory-{2,3}-*.md`).
- `docs/stdlib-roadmap.adoc` — promote `Sqlite.affine` from `partial /
Coverage audit needed` to `usable`, and add `Aggregate.affine` +
`Transaction.affine` rows to the inventory snapshot.
- `.machine_readable/6a2/STATE.a2ml` — bump `last-updated` to
`2026-06-02` (this file mirrors, it does not lead — per its own
drift-flag).

## What was deliberately *not* touched

- `docs/ROADMAP.adoc` — language/compiler progress; the db-theory PRs
are stdlib-side and don't move the language roadmap.
- A `TEST-NEEDS` file — none exists at repo root; the three PRs each
shipped their own Deno-ESM smoke harness under `tests/codegen-deno/`
(`sqlite_introspect_bulk` / `transaction_smoke` / `aggregate_smoke`) and
the AOT gate auto-picked the new modules up.
- License / SPDX headers — per the estate's no-automated-licence-edits
rule.

## Test plan

- [x] `bash tools/check-doc-truthing.sh` passes locally — banner /
primacy / mirror invariants intact, over-claim ratchet unchanged (no new
baseline entry needed).
- [x] Commit GPG-signed; verified `gpg: Good signature`.

Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant