Skip to content

Add SQL Server, Spanner and DuckDB to goldeneye: generate their dialects and check the analyze cases against live databases - #4619

Merged
kyleconroy merged 3 commits into
mainfrom
claude/goldeneye-mssql-spanner-l0am6h
Sep 13, 2026
Merged

kyleconroy merged 3 commits into
mainfrom
claude/goldeneye-mssql-spanner-l0am6h

Conversation

@kyleconroy

Copy link
Copy Markdown
Collaborator

Summary

Three more engines get goldeneye's two checks: their generated dialect files are verified byte for byte against a live database, and their analyze_* cases are verified against what the database itself reports.

  • SQL Server (mssql, MSSQL_SERVER_URI, the mcr.microsoft.com/mssql/server:2025-latest image). relations.jsonl is every view of sys and INFORMATION_SCHEMA, each described by sys.dm_exec_describe_first_result_set from a scratch user database, since master lists internal views no query can name. The analyze check describes each query without running it: result columns from the same DMV, parameter types from sp_describe_undeclared_parameters, and the column a parameter is compared with or assigned to from the estimated XML showplan compiled with the parameters declared as those types. Spellings types.jsonl lists as aliases (numeric, timestamp) are reported by the dialect's name (decimal, rowversion).
  • Spanner (spanner, SPANNER_SERVER_URI, the us-docker.pkg.dev/spanner-omni/images/spanner-omni:2026.r2.1-beta image, writing into the googlesql dialect). relations.jsonl is every view of INFORMATION_SCHEMA and SPANNER_SYS. The analyze check creates a database per case with the schema as DDL, writes the fixture, compiles each query in PLAN mode (DML in a rolled-back read-write transaction), and reads columns and parameter types from the result metadata and provenance from the query plan: Serialize Result lists the outputs, Scan nodes name table columns, comparisons show up as ($col = @param), and a DML plan lists its THEN RETURN outputs before the written values.
  • DuckDB (duckdb). 2.0 has no release yet, so install duckdb downloads the current build of DuckDB's v2.0 preview channel (a rolling tarball; nothing can be pinned, duckdb.GeneratedFrom records the build the dialect came from) into the user cache directory, where Locate finds it after DUCKDB and before PATH. The dialect is regenerated from that build, which adds 21 functions. The new analyze check drives the CLI: parameter types from EXPLAIN (FORMAT json) EXECUTE with a string sentinel bound to each parameter, result types from DESCRIBE with a typed NULL in each parameter's place, provenance from the query text resolved against duckdb_columns() (DuckDB prints plans with bare column names and describes no DML), and expression nullability by running the query with a value of each parameter's type over the fixture and over no rows.

Each engine gets a gen.yml job, a docker-compose.yml service where a server is needed, tests that skip without a database, and a README section describing what the database says and what it keeps to itself.

What the checks found

  • A rowversion or sysname column is NOT NULL unless declared nullable; the SQL Server converter now does that.
  • SQL Server rejects CAST to an alias type and = on vector values, and its describe function calls every computed column nullable. analyze_types/mssql now casts nullable columns and moves the parameter casts into the WHERE clause; its golden is regenerated.
  • Spanner requires STRING(MAX) and rejects NUMERIC(p,s), BIGNUMERIC, GEOGRAPHY, INTERVAL and STRUCT columns, and cannot return a STRUCT as a column. The three simple GoogleSQL cases get STRING(MAX); the BigQuery-flavoured types case is kept intact as analyze_types/bigquery, which the Spanner check does not read, beside a Spanner-valid analyze_types/googlesql.
  • SPANNER_SYS has a PROTO column, so the GoogleSQL types.jsonl gains a proto type.
  • The new DuckDB build lists each type alias once per schema it is visible in; the generator folds them.

The last commit fixes what an adversarial review of the three packages found (Spanner's DML output order and a nil Close, DuckDB's conversion-error matching and cast-type reading, SQL Server's ? numbering and block-aware statement splitting, among others).

Testing

  • go test ./... in internal/goldeneye against a live SQL Server 2025, Spanner Omni 2026.r2.1 and DuckDB v2.0.0-alpha41396: the three dialects match, and all 17 analyze cases (8 MSSQL, 4 GoogleSQL, 5 DuckDB) match byte for byte.
  • TestReplay/base for every case in the main module passes.
  • The gen.yml jobs themselves were not exercised.

Known limits

  • Spanner reports nothing about the nullability of an expression, so the Spanner check reports an expression as not null.
  • Spanner's view names are kept in upper case; Spanner matches any case, but sqlc's GoogleSQL engine matches names as spelled.
  • The DuckDB gen job goes red whenever the preview channel moves past the recorded build, which is the intended signal until 2.0 is released and can be pinned with a checksum.

🤖 Generated with Claude Code

https://claude.ai/code/session_01F7cPsawATXMfiYqWg8nBVb


Generated by Claude Code

Add two engines to goldeneye, each generating relations.jsonl from a live
database and checking the engine's analyze cases against it.

mssql reads a SQL Server named by MSSQL_SERVER_URI: relations.jsonl is
every view of sys and INFORMATION_SCHEMA as the server describes a
SELECT * from it, listed from a scratch user database. The analyze check
describes each query without running it: result columns from
sys.dm_exec_describe_first_result_set, parameter types from
sp_describe_undeclared_parameters, and the column a parameter is
compared with or assigned to from the estimated showplan.

spanner reads a Spanner Omni server named by SPANNER_SERVER_URI, the
gRPC endpoint of the spanner-omni container image, and writes into the
googlesql dialect: relations.jsonl is every view of INFORMATION_SCHEMA
and SPANNER_SYS. The analyze check creates a database per case and
compiles each query in PLAN mode, taking columns and parameter types
from the result metadata and provenance from the query plan.

Both engines get a gen workflow job, a docker-compose service and a
README section. Generating the dialects and running the checks turned
up a few things in sqlc and its cases:

- A rowversion or sysname column is NOT NULL unless declared nullable,
  which the SQL Server converter now does.
- SQL Server rejects CAST to an alias type and a comparison of vector
  values, and Spanner requires a length on STRING, rejects NUMERIC(p,s),
  BIGNUMERIC, GEOGRAPHY, INTERVAL and STRUCT columns and cannot return a
  STRUCT as a column, so the analyze cases are rewritten to what each
  database accepts; the BigQuery-flavoured types case is kept as
  analyze_types/bigquery, which the Spanner check does not read.
- The GoogleSQL dialect gains a proto type, which SPANNER_SYS has a
  column of.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7cPsawATXMfiYqWg8nBVb
DuckDB 2.0 has no release to download yet, so `install duckdb` fetches
the current build of DuckDB's v2.0 preview channel, a rolling tarball
per platform with no per-build download and no checksum to pin, into
the user cache directory, where Locate finds it after DUCKDB and before
PATH. GeneratedFrom records the build the dialect was generated from,
and the dialect is regenerated from the channel's current build, which
adds a few functions and lists each type alias once per schema it is
visible in, which the generator now folds.

The duckdb engine gains an analysis check over the DuckDB analyze cases.
The CLI reports a parameter's type through the unoptimized logical plan
of the prepared query explained with a string sentinel bound to each
parameter, and a result column's name and type through DESCRIBE with a
typed NULL in each parameter's place; it prints every column by its bare
name, so which table a result column is read from and which column a
parameter stands in for come from the query text resolved against the
catalog, and since it tracks no nullability of expressions, the query is
run over the fixture and over no rows to see which columns come back
NULL. A spelling the generated types.jsonl lists as an alias, such as
json for varchar, is reported by the dialect's name for it.

The gen workflow gets a duckdb job that installs and generates from the
channel, and the README and CLAUDE.md describe the engine.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7cPsawATXMfiYqWg8nBVb
SQL Server: a schema is split into statements outside BEGIN ... END and
CASE ... END blocks, so a trigger or procedure body keeps its semicolons,
and a GO on the first line is honoured; each ? becomes a parameter of
its own rather than every ? the same one; the catalog's columns are
matched to the described ones by name rather than position; Analyze
refuses a server of another major release the way Generate does; the
views are ordered by the bytes of their names rather than the server's
collation; the plan walk's cycle guard is scoped to the path; and the
generator's comment no longer claims a canonicalization it does not do.

Spanner: Close no longer dereferences a nil client when opening fails
half way; a DML statement is recognised past a leading comment; a DML
plan's outputs are read as the THEN RETURN columns followed by the
written values, which is the order Omni prints, and a returned
expression is no longer taken for the column it is named after; an
UPDATE's SET list is read with quotes and parentheses honoured up to the
last top-level WHERE; a case's database is named with a hash of the
case, so two cases with the same head stay apart; NotFound is checked
by status code; a failed fixture write is rolled back; @@variables are
not parameters; and PROTO and ENUM types are spelled one way.

DuckDB: sentinel types are read from EXPLAIN (FORMAT json), whose
expressions are strings, rather than from the box the CLI draws, and a
sentinel the plan prints bare is a VARCHAR; a conversion error that
names a sentinel in any form, or none, rebinds a parameter to NULL; the
type after a :: or CAST stops at anything but a multi-word type's
words; a CTE that shares a table's name is not the table; a parameter
inside a subquery takes the binder's type rather than a partner from
the outer scope; parameters are replaced token-wise, leaving string
literals alone; the value a parameter is bound to is chosen by its type
as spelled, so a JSON parameter gets a JSON value; and two enums with
the same labels resolve to the same name every run.

The MSSQL converter also treats a schema-qualified sys.sysname as NOT
NULL by default.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7cPsawATXMfiYqWg8nBVb
@kyleconroy
kyleconroy merged commit bdbe55d into main Sep 13, 2026
12 checks passed
@kyleconroy
kyleconroy deleted the claude/goldeneye-mssql-spanner-l0am6h branch September 13, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants