sqlite: add virtual table support via createModule()

Expose SQLite's virtual table API through a new
`database.createModule(name, options)` method, wrapping
`sqlite3_create_module_v2()`. This enables read-only virtual tables
backed by JavaScript data sources, usable either as an eponymous table
(`SELECT * FROM module_name`) or via `CREATE VIRTUAL TABLE t USING
module_name`. Hidden columns pass parameters using table-valued
function syntax (`SELECT * FROM module_name(param1, param2)`).

`options` accepts `columns`, `rows`, `directOnly`, and
`useBigIntArguments`. Column types are validated against INTEGER,
TEXT, REAL, BLOB, and ANY, and column names are quoted when building
the `sqlite3_declare_vtab()` schema.

Rebased from #61544, which was
opened by byteforge38 and became inactive. Changes on top of that
work:

- xColumn reports the value each hidden column was constrained to,
  rather than NULL. SQLite treats xBestIndex's `omit` as a hint, so it
  may recheck a constraint it already handed to xFilter; against NULL
  that recheck rejected every row, and `gs(1, 3) WHERE start = 1`
  returned no rows.
- xBestIndex lowers estimatedCost as it consumes constraints. With a
  constant cost the planner was free to pick the unconstrained plan and
  recheck afterwards, so a correlated parameter such as
  `FROM t, gs(t.a, t.a + 1)` also returned no rows.
- Violations of the iteration protocol report a SQLite error instead of
  calling PropagateJSError with no JavaScript exception pending. That
  left `.all()` returning undefined and `exec()` reporting success.
- xBestIndex passes the constrained hidden-column indices to xFilter
  through idxStr rather than an int bitmask, which previously aliased
  for parameter indices at or above the width of an int.
- xFilter, xNext, and xColumn take a CallbackDepthGuard. Without it
  close() from inside rows(), an iterator's next(), or a row getter
  finalized the statement that SQLite was still stepping, crashing the
  process.
- xClose calls the iterator's return() method so generator `finally`
  blocks run when SQLite stops stepping early, as it does for LIMIT or
  a `break` out of a for...of loop. It is skipped while tearing down
  from ~StatementSync or ~DatabaseSync, which run from garbage
  collection callbacks where JavaScript cannot be executed; an
  abandoned generator does not run `finally` in JavaScript either. It
  is also skipped when an error is already pending, so that error still
  reaches the caller.
- VirtualTableModule holds a BaseObjectWeakPtr<DatabaseSync> to match
  UserDefinedFunction instead of a raw pointer.
- createModule() rejects being called from an authorizer callback.
- Documents that values yielded by rows() follow the usual conversion
  rules, so a number is stored as REAL and a BigInt as INTEGER even
  when a column declares INTEGER, since virtual tables do not apply
  column affinity to the values they return.

Refs: #61544
Refs: #63826
Fixes: #61539

Co-authored-by: byteforge38 <stormcraft318@gmail.com>
Signed-off-by: Trevor Burnham <trevorburnham@gmail.com>
Assisted-by: Claude Opus 5
PR-URL: #65787
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>

Author: 2 people

doc/api/sqlite.md

@@ -860,6 +860,114 @@ console.log(allUsers);
 // ]
 ```
 
+### `database.createModule(name, options)`
+
+<!-- YAML
+added: REPLACEME
+-->
+
+* `name` {string} The name of the virtual table module. This name is used in
+  `CREATE VIRTUAL TABLE ... USING name` statements and as an eponymous table
+  name.
+* `options` {Object} Module configuration settings.
+  * `columns` {Array} An array of column definitions. Each element is an object
+    with the following properties:
+    * `name` {string} The name of the column.
+    * `type` {string} The declared type of the column. Must be one of
+      `'INTEGER'`, `'TEXT'`, `'REAL'`, `'BLOB'`, or `'ANY'`.
+    * `hidden` {boolean} If `true`, the column is hidden and acts as a
+      parameter for table-valued function usage. **Default:** `false`.
+  * `rows` {Function} A function called to produce rows when the virtual table
+    is queried. The function receives values for hidden columns (parameters) as
+    arguments, in the order they are defined. Must return an iterable (such as
+    an array or generator) where each element is an array of column values.
+  * `directOnly` {boolean} If `true`, the virtual table can only be used in
+    top-level SQL statements and cannot be used inside triggers or views.
+    **Default:** `false`.
+  * `useBigIntArguments` {boolean} If `true`, integer parameters passed to
+    `rows` are converted to `BigInt`s. **Default:** `false`.
+
+Registers a virtual table module with the database. This method is a wrapper
+around [`sqlite3_create_module_v2()`][]. Virtual tables allow JavaScript code
+to provide the backing data for SQL tables. The registered module can be used
+in two ways:
+
+* **Eponymous table**: Query the module name directly without creating a table
+  (e.g., `SELECT * FROM module_name`).
+* **Named virtual table**: Use `CREATE VIRTUAL TABLE t USING module_name` to
+  create a persistent virtual table.
+
+Hidden columns can be used to pass parameters to the `rows` function using
+table-valued function syntax (e.g., `SELECT * FROM module_name(param1, param2)`).
+
+Values yielded by `rows` follow the conversion rules in [Type conversion between
+JavaScript and SQLite][]: a {number} is stored as `REAL` and a {bigint} is
+stored as `INTEGER`, regardless of the column's declared `type`. Unlike an
+ordinary table, a virtual table does not apply column affinity to the values it
+returns, so yield a {bigint} when a column needs `INTEGER` storage:
+
+```js
+db.createModule('counter', {
+  columns: [{ name: 'value', type: 'INTEGER' }],
+  *rows() {
+    yield [1];   // typeof(value) is 'real'
+    yield [2n];  // typeof(value) is 'integer'
+  },
+});
+```
+
+```cjs
+const { DatabaseSync } = require('node:sqlite');
+
+const db = new DatabaseSync(':memory:');
+
+db.createModule('generate_series', {
+  columns: [
+    { name: 'value', type: 'INTEGER' },
+    { name: 'start', type: 'INTEGER', hidden: true },
+    { name: 'stop', type: 'INTEGER', hidden: true },
+    { name: 'step', type: 'INTEGER', hidden: true },
+  ],
+  *rows(start, stop, step) {
+    start ??= 0;
+    stop ??= 10;
+    step ??= 1;
+    for (let i = start; i <= stop; i += step) {
+      yield [i];
+    }
+  },
+});
+
+console.log(db.prepare('SELECT * FROM generate_series(1, 5, 1)').all());
+// Prints: [ { value: 1 }, { value: 2 }, { value: 3 }, { value: 4 }, { value: 5 } ]
+```
+
+```mjs
+import { DatabaseSync } from 'node:sqlite';
+
+const db = new DatabaseSync(':memory:');
+
+db.createModule('generate_series', {
+  columns: [
+    { name: 'value', type: 'INTEGER' },
+    { name: 'start', type: 'INTEGER', hidden: true },
+    { name: 'stop', type: 'INTEGER', hidden: true },
+    { name: 'step', type: 'INTEGER', hidden: true },
+  ],
+  *rows(start, stop, step) {
+    start ??= 0;
+    stop ??= 10;
+    step ??= 1;
+    for (let i = start; i <= stop; i += step) {
+      yield [i];
+    }
+  },
+});
+
+console.log(db.prepare('SELECT * FROM generate_series(1, 5, 1)').all());
+// Prints: [ { value: 1 }, { value: 2 }, { value: 3 }, { value: 4 }, { value: 5 } ]
+```
+
 ### `database.createSession([options])`
 
 <!-- YAML
@@ -1968,6 +2076,7 @@ callback function to indicate what type of operation is being authorized.
 [`sqlite3_column_origin_name()`]: https://www.sqlite.org/c3ref/column_database_name.html
 [`sqlite3_column_table_name()`]: https://www.sqlite.org/c3ref/column_database_name.html
 [`sqlite3_create_function_v2()`]: https://www.sqlite.org/c3ref/create_function.html
+[`sqlite3_create_module_v2()`]: https://www.sqlite.org/c3ref/create_module.html
 [`sqlite3_create_window_function()`]: https://www.sqlite.org/c3ref/create_function.html
 [`sqlite3_db_filename()`]: https://sqlite.org/c3ref/db_filename.html
 [`sqlite3_deserialize()`]: https://sqlite.org/c3ref/deserialize.html