test: add release-validation coverage for post-v0.8.7 merged PRs#952
Merged
Conversation
Adds regression tests for the 8 PRs merged since `v0.8.7`, authored during
pre-release validation. Two independent authoring passes (suffix `-codex.test.ts`
from the codex CLI track, plus `release-validation/*.test.ts` from the Claude
multi-agent track) intentionally overlap to maximize edge-case coverage.
opencode (`packages/opencode/test/release-validation/`):
- `question-937*` non-interactive question tool: `ALTIMATE_NON_INTERACTIVE`
/ `ALTIMATE_FORCE_INTERACTIVE` / `ALTIMATE_AUTO_ANSWER`
matrix, output-text contract, `run.ts`/`bash.ts` env guards
- `mcp-datamate-893*` MCP config normalize/merge, enabled-state persistence,
recursive `**/mcp.json` discovery, datamate transport selection
- `session-transcript-941*` transcript REST endpoint: query coercion, content negotiation,
tool/thinking detail gating, error schema
- `serve-upgrade-940*` headless `serve` startup upgrade check: scheduling, failure
isolation, version-compare boundaries
- `serve-trace-log-929*` trace-directory startup logging
- `chunk-timeout-844*` `DEFAULT_CHUNK_TIMEOUT` SSE watchdog behavior
- `windows-installer-930*` `install.ps1` static analysis (HTTPS URLs, error handling,
PATH safety, idempotency, no secret leakage) + win32 dispatch
dbt-tools (`packages/dbt-tools/test/`):
- `dbt-cli-release-validation.test.ts` #933 error-bubbling: malformed JSON, exit-code
redaction, ANSI stripping, no-signal fallback
- `dbt-cli-extra-codex.test.ts` result-shape, `--limit` boundaries, last-error
selection, inline-compile error paths
All new tests pass and are typecheck-clean. 4 `test.todo` markers document real,
pre-existing edge cases surfaced during review (transcript path-traversal -> 500,
`normalizeMcpConfig` dropping `updatedAt`, `install.ps1` missing archive checksum,
`execDbtCompile` stale-manifest fallback). No source files are modified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (14)
Disabled knowledge base sources:
📝 WalkthroughWalkthroughAdds thirteen new Bun test files as release-validation regression coverage for eight previously merged PRs ( Changesdbt-cli regression tests (PR
Provider SSE chunk-timeout regression tests (PR
MCP/Datamate IDE-aware transport and config sync regression tests (PR
QuestionTool non-interactive and auto-answer regression tests (PR
Serve startup tracing and upgrade-check regression tests (PRs
Session transcript endpoint regression tests (PR
Windows PowerShell installer regression tests (PR
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
ralphstodomingo
added a commit
that referenced
this pull request
Jun 18, 2026
The #952 release-validation suite asserted the exact #930 URL literals. This PR builds the archive and checksums.txt from a shared $base (so they always come from the same release), so update those assertions to the $base/$url form, and convert the now-obsolete "verification deferred" test.todo into a real assertion that Test-Checksum verifies SHA256 before extraction. (This test never ran on this PR until it was retargeted from the merged feat/windows-powershell-installer branch to main.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG
ralphstodomingo
added a commit
that referenced
this pull request
Jun 18, 2026
The #952 release-validation suite asserted the latest path hard-fails with "Failed to fetch version information" (>=2) and that exit 1 appears >=3 times. This PR makes the latest path retry then degrade gracefully instead of aborting, so update those assertions: the latest path no longer hard-fails (the unsupported -arch and pinned-404 paths still exit 1, hence >=2). (This test never ran on this PR until it was retargeted from the merged feat/windows-powershell-installer branch to main.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG
sahrizvi
pushed a commit
that referenced
this pull request
Jun 18, 2026
) * feat(install): verify release archive checksums (both installers) Raises the integrity bar for the standalone installers (follow-up to #930). - release.yml: generate a checksums.txt (sha256sum format) over the release archives and publish it as a release asset. - install (bash) + install.ps1: fetch checksums.txt and verify the downloaded archive's SHA256 before extracting. Hard-fail on mismatch; soft-skip with a notice when checksums.txt is absent (older pinned releases) or unreachable, so existing version-pinned installs keep working. - Cross-platform sha in bash (sha256sum or shasum -a 256); Get-FileHash on Windows. Verification runs before extraction in both. - Tests: checksum-verification.test.ts asserts release.yml publishes the file and both installers fetch + compare + hard-fail on mismatch. Verified: bash -n clean; install.ps1 parses clean and the Pester suite (6/6) still passes on PowerShell 7.6.2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(install): address checksum-verification review - install.ps1: decode a Byte[] checksums.txt body so verification works on Windows PowerShell 5.1. GitHub serves release assets as octet-stream, so on PS 5.1 Invoke-WebRequest returns .Content as Byte[]; it coerced to a decimal string and every check silently soft-skipped (sahrizvi, P1). - install.ps1: pin the archive and checksums.txt to the resolved release tag instead of the mutable latest/ URL, so a release published mid-install can't hand back mismatched assets and trigger a spurious hard-fail (cubic, P2). Falls back to latest/ only when the version can't be resolved. - install: in verify_checksum, clean up via $(dirname "$file") rather than the caller's dynamically-scoped $tmp_dir local — self-contained (cubic, P2). - tests: Pester coverage for Test-Checksum (String + Byte[] + mismatch paths, verified to fail without the decode) and TS guards for the decode and the PowerShell same-release pinning. Note: the bash installer is intentionally left on the latest/download path here to keep this PR disjoint from #946 (which owns the bash latest-version block); the two PRs then merge in either order with no conflict. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG * fix(install.ps1): ASCII-only so it parses on Windows PowerShell 5.1 install.ps1 had no BOM and used a few non-ASCII characters (em dash, ellipsis, right arrow) in comments and messages. Windows PowerShell 5.1 — the default shell on Windows 10 and preinstalled on Windows 11 — reads a BOM-less file as the system ANSI codepage, not UTF-8, so those multi-byte characters corrupt the token stream and the whole script fails to parse (verified on real PS 5.1: "The '<' operator is reserved", cascading to "Missing closing '}'"). This is a pre-existing issue (the characters predate this PR) that CI doesn't catch because the Pester job runs under pwsh (PowerShell 7, UTF-8 by default). Replacing the three characters with ASCII equivalents (-, ..., ->) makes the installer parse and run on PS 5.1 while keeping pwsh behavior identical. Verified end-to-end on real Windows PowerShell 5.1: resolve version -> download -> extract -> place the binary all succeed. Same transliteration is applied verbatim in #946 so the two PRs merge cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG * fix(install): guard the verify_checksum cleanup against a pathological path Defensive depth (coderabbit): only `rm -rf` the cleanup dir when dirname resolves to a real subdirectory, never "." or "/", so an unexpectedly empty or root-level $file can't wipe the cwd or worse. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG * test: update #930 release-validation for the checksum URL refactor The #952 release-validation suite asserted the exact #930 URL literals. This PR builds the archive and checksums.txt from a shared $base (so they always come from the same release), so update those assertions to the $base/$url form, and convert the now-obsolete "verification deferred" test.todo into a real assertion that Test-Checksum verifies SHA256 before extraction. (This test never ran on this PR until it was retargeted from the merged feat/windows-powershell-installer branch to main.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: ralphstodomingo <ralphstodomingo@users.noreply.github.com>
sahrizvi
pushed a commit
that referenced
this pull request
Jun 18, 2026
* fix(install): don't hard-fail when the GitHub releases API blips Reported on #930: a transient 504 from api.github.com/.../releases/latest (or the 60/hr/IP unauthenticated rate limit) aborted the whole install with "Failed to fetch version information" — even though the download itself uses releases/latest/download/<file>, which GitHub resolves server-side with no API call. The API response only feeds the version-string display and the already-installed short-circuit. Both installers now, in the latest path: - retry the API call up to 3x with linear backoff (bash uses curl --fail so a 504 retries instead of parsing an error body); - on continued failure, print a muted notice and proceed to install latest anyway (version string shown as "latest"); - only short-circuit as "already installed" on a real version match — never treat empty==empty (unresolved version + unreadable binary) as installed. Pinned-version installs (-Version / --version) are unchanged: a genuine 404 still hard-fails. Tests: version-fetch-resilience.test.ts pins the retry + graceful-degrade behavior in both installers. bash -n clean; install.ps1 parses clean and the Pester suite (6/6) still passes on PowerShell 7.6.2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(install): address latest-version-fetch review - install: append `|| true` to the retry's curl|sed assignment. Under `set -euo pipefail` a failing `curl --fail` propagated through the pipeline and aborted the script at attempt 1, before the loop could retry or degrade (sahrizvi; reproduced: exit 22 without the fix, all 3 attempts + degrade with it). Also add `--max-time 10` to bound a dead-air socket. - install.ps1: reset $specificVersion to $null (not "") on the degrade path, so the already-installed short-circuit can't false-match "" -eq "" when the version probe of a missing/corrupt binary also yields "" (dev-punia, sahrizvi). - install.ps1: add -TimeoutSec 10 to Invoke-RestMethod (defaults to 100s on PS 5.1, unbounded on PS 7+) to bound retries on dead air (sahrizvi). - tests: TS guards for `|| true`, --max-time/-TimeoutSec, and the $null reset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG * fix(install.ps1): ASCII-only so it parses on Windows PowerShell 5.1 install.ps1 had no BOM and used a few non-ASCII characters (em dash, ellipsis, right arrow) in comments and messages. Windows PowerShell 5.1 - the default shell on Windows 10 and preinstalled on Windows 11 - reads a BOM-less file as the system ANSI codepage, not UTF-8, so those multi-byte characters corrupt the token stream and the whole script fails to parse (verified on real PS 5.1). This is a pre-existing issue (the characters predate this PR) that CI doesn't catch because the Pester job runs under pwsh (PowerShell 7, UTF-8 by default). Replacing the three characters with ASCII equivalents (-, ..., ->) makes the installer parse and run on PS 5.1 while keeping pwsh behavior identical. Also removes the now-obsolete "integrity verification deferred" NOTE comment: the sibling PR #942 implements that verification and removes the same block, so deleting it here too keeps the two PRs mergeable in either order with no conflict. Same transliteration is applied verbatim in #942. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG * test: update #930 release-validation for resilient version fetch The #952 release-validation suite asserted the latest path hard-fails with "Failed to fetch version information" (>=2) and that exit 1 appears >=3 times. This PR makes the latest path retry then degrade gracefully instead of aborting, so update those assertions: the latest path no longer hard-fails (the unsupported -arch and pinned-404 paths still exit 1, hence >=2). (This test never ran on this PR until it was retargeted from the merged feat/windows-powershell-installer branch to main.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019M7GkS3bYZaFhEbBhVTecG --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: ralphstodomingo <ralphstodomingo@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PINEAPPLE
What does this PR do?
Adds regression test coverage for the 8 PRs merged since
v0.8.7, authored during pre-release validation. No source files are modified — tests only.Two independent authoring passes intentionally overlap to maximize edge-case coverage:
*-codex.test.ts— codex CLI trackrelease-validation/*.test.ts— Claude multi-agent trackopencode (
packages/opencode/test/release-validation/):question-937*(fix: auto-resolve question tool in non-interactive contexts #937) — non-interactive question tool:ALTIMATE_NON_INTERACTIVE/ALTIMATE_FORCE_INTERACTIVE/ALTIMATE_AUTO_ANSWERmatrix, output-text contract,run.ts/bash.tsenv guardsmcp-datamate-893*(feat: IDE-aware datamate transport, enabled-state persistence, and /mcps command #893) — MCP config normalize/merge, enabled-state persistence, recursive**/mcp.jsondiscovery, datamate transport selectionsession-transcript-941*(feat: [AI-6949] add session transcript REST endpoint for datamates extension #941) — transcript REST endpoint: query coercion, content negotiation, tool/thinking detail gating, error schemaserve-upgrade-940*(fix: trigger auto-update check on headlessservestartup #940) — headlessservestartup upgrade check: scheduling, failure isolation, version-compare boundariesserve-trace-log-929*(serve: log trace output directory on startup #929) — trace-directory startup loggingchunk-timeout-844*(fix: increase DEFAULT_CHUNK_TIMEOUT from 2min to 5min #844) —DEFAULT_CHUNK_TIMEOUTSSE watchdog behaviorwindows-installer-930*(feat: Windows PowerShell installer (install.ps1) #930) —install.ps1static analysis (HTTPS URLs, error handling, PATH safety, idempotency, no secret leakage) + win32 dispatchdbt-tools (
packages/dbt-tools/test/):dbt-cli-release-validation.test.ts(fix: bubble real dbt show error instead of generic "Could not parse" #933) — error-bubbling: malformed JSON, exit-code redaction, ANSI stripping, no-signal fallbackdbt-cli-extra-codex.test.ts(fix: bubble real dbt show error instead of generic "Could not parse" #933) — result shape,--limitboundaries, last-error selection, inline-compile error paths4
test.todomarkers document real, pre-existing edge cases surfaced during review (do not block release): transcript path-traversal → 500,normalizeMcpConfigdroppingupdatedAt,install.ps1missing archive checksum,execDbtCompilestale-manifest fallback.Type of change
Issue for this PR
Closes #951
How did you verify your code works?
bun turbo typecheck— green (all 7 packages, forced/no-cache)cd packages/opencode && bun test test/release-validation/— 172 pass / 3 todo / 0 fail (12 files)cd packages/dbt-tools && bun test test/dbt-cli-extra-codex.test.ts test/dbt-cli-release-validation.test.ts— 18 pass / 1 todo / 0 failcd packages/dbt-tools && bun run test(full suite, regression check) — 82 pass / 0 failChecklist
Summary by cubic
Adds regression tests for the 8 PRs merged since
v0.8.7to strengthen release validation; no source files are changed. Addresses #951. Two overlapping tracks (*-codex.test.tsandrelease-validation/*.test.ts) maximize edge-case coverage.packages/opencode: non-interactivequestiontool env matrix and output contract; MCP/datamate config normalize/merge, discovery, and transport selection; session transcript API (query coercion, content negotiation, detail gating, error schema); serve startup upgrade check (scheduling, failure isolation, version compare) and trace-dir logging; provider SSE chunk-timeout default; Windowsinstall.ps1safety and native win32 upgrade dispatch.packages/dbt-tools:dbt-clierror bubbling (malformed JSON, exit-code redaction, ANSI stripping, no-signal fallback) and result shape/--limit/inline-compile paths.4 test.todo markers document known, pre-existing gaps and do not block the release.
Written for commit a8a08c6. Summary will update on new commits.
Summary by CodeRabbit