Skip to content

Move scoped introspection benchmark to TypeScript and keep results local - #1851

Merged
Zetazzz merged 1 commit into
mainfrom
scoped-benchmark-typescript-430c2a16
Sep 21, 2026
Merged

Zetazzz merged 1 commit into
mainfrom
scoped-benchmark-typescript-430c2a16

Conversation

@Zetazzz

@Zetazzz Zetazzz commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

The scoped introspection benchmark from #1823 kept one run's CSV/environment JSON in source control and used a CommonJS script that required an absolute checkout path. Move the runner to src/benchmarks/scoped-introspection/run.ts, compiled by the existing CJS/ESM build, and expose pnpm --filter @constructive-io/perf-harness scoped:introspection with an optional --output directory. The stock/scoped experiment, validation, SQL diagnostics, and database cleanup are preserved.

Remove the recorded CSV and environment JSON, ignore the default generated-results directory, and retain the dated methodology, historical performance summary, environment details, and links to the original artifacts at their fixed commit. The new validation results remain local; the historical performance figures are unchanged.

Validation:

  • Frozen-lockfile install and CJS/ESM builds passed.
  • All 100 perf-harness tests passed, including import safety and CLI argument/output-path checks.
  • Ran the compiled entrypoint against local PostgreSQL: all 42 measured samples across 0/10/50 unrelated schemas passed schema/runtime validation; SQL plans were written and the temporary database was dropped.
  • Focused ESLint and git diff --check passed.

@tenki-reviewer

tenki-reviewer Bot commented Sep 21, 2026

Copy link
Copy Markdown

Review complete. No issues found — approved ✅.


This PR introduces packages/perf-harness, a performance-harness package focused on measuring scoped vs stock introspection cost. The core is src/benchmarks/scoped-introspection/run.ts, which executes both introspection variants against a seeded catalog fixture, captures environment/Postgres metadata before and after runs, computes median timings over warmup+measured repetitions, and writes results. A benchmarks/scoped-introspection.cjs provides the runnable entry, backed by environment and sample CSV fixtures, a written-up benchmark report, and a .gitignore for generated results.

Files Change
src/benchmarks/scoped-introspection/run.ts Implements the scoped vs stock introspection benchmark driver, sampling, seeding, and result persistence.
benchmarks/scoped-introspection.cjs + docs/env/samples Provides the runnable entrypoint and its environment, sample CSV, and markdown report fixtures.
tests/entrypoints.test.ts Verifies the package's CLI/entrypoint wiring.
package.json, README.md, .gitignore Scaffold the new package, document usage, and ignore generated result artifacts.

Review of the changed lines surfaced no correctness, security, concurrency, or performance defects that met the confidence threshold after adjudication; the candidate observations were rejected as not actionable within the diff.

Reviewed commit: 4fd8f7b

@Zetazzz
Zetazzz merged commit 2052f6f into main Sep 21, 2026
21 checks passed
@Zetazzz
Zetazzz deleted the scoped-benchmark-typescript-430c2a16 branch September 21, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant