fix(ship): scope fallow audit to the ship's actual base, not main-autodetect (DK-5) by norvalbv · Pull Request #98 · norvalbv/devkit

norvalbv · 2026-07-17T07:58:05Z

DK-5 — `ship --base` audits against `main`, not the real ship base

devkit ship --base  cuts the ephemeral gate worktree from 's origin tip, but the in-chain
fallow audit (both the self-host advisory fragment and the overlay gate) always ran bare —
fallow audit, no --base — so fallow auto-detected main as its comparison ref regardless of what
the worktree was actually cut from. Shipping --base feat/studio from a finalized checkout, where
feat/studio is itself a long-lived stack with its own pre-existing (not-yet-merged) dead-code /
duplication findings vs main, misreported those as "new" and failed the audit.

Fallow's own audit command already grandfathers correctly when given the right ref: its gate: new-only default runs the base ref too and marks inherited findings introduced=false. The fix is
to thread the exact commit ship already resolves the worktree from — through as DEVKIT_SHIP_BASE_SHA
(exported by ship-branch.sh/reship.sh, alongside the existing DEVKIT_SHIP_MODE) — so devkit's
own fallow fragments pass fallow audit --base "$DEVKIT_SHIP_BASE_SHA" when it's set, falling back to
fallow's own default on a plain git commit. This is unconditional (not just under --base): even
the default ship case is more precise than main-autodetection for any branch that isn't a fresh cut
off main.

.husky/pre-commit (this repo's own self-hosted hook) is regenerated in the same commit — required,
since self-host.test.mts's parity test compares it byte-for-byte against the generator.

A narrower, separate bug (link-gate-configs.sh symlinking fallow-baselines/ from the invoking
checkout rather than the ship base — relevant only to consumers using fallow's own self-installed
git hook) is deliberately out of scope here, mirroring how DK-3 split an analogous baseline-follows-
ROOT bug into its own PR.

Also in this PR

.husky/pre-push runs the full, untargeted test suite on every push, so any pre-existing failure
blocks shipping anything. overlay.test.mts's workRepo()-based tests call applyInit in-process
(unlike the ship-branch/reship/reconcile suites, which spawn devkit as a subprocess and isolate git
via an explicit per-call env), so its internal execFileSync('git', …) calls (e.g.
installHealAlias's alias.ci collision check) inherited whatever process.env already was —
including a developer machine's own real ~/.gitconfig. overlay.mts deliberately never clobbers an
existing git ci alias, so a machine with one already set makes that check correctly skip
installing devkit's self-heal alias, which then read as a false test failure. Isolated
GIT_CONFIG_GLOBAL/GIT_CONFIG_SYSTEM to /dev/null for the file's duration (restored after),
matching the isolation every sibling git-integration test file already does. Unrelated to DK-5 itself,
but bundled in since it's what blocked the push.

Verification

bun run test:run -- cli/__tests__/ship-branch.test.mts cli/__tests__/reship.test.mts cli/__tests__/self-host.test.mts cli/__tests__/husky-block.test.mts cli/__tests__/overlay.test.mts — 153 pass
bun run typecheck clean
bun run lint clean
bun run lint:structure clean
git diff .husky/pre-commit confirmed scoped to only the # devkit:fallow-advisory fragment

Also bundled: a one-line formatting fix

origin/main advanced (PR #94) between this branch's creation and shipping; gate-engine/review/ run-review.mts's savePasses(...) call in that merge violates biome's line-length rule. Since
bun run lint (biome check .) scans the whole repo, this now blocks any ship off main — bundled
the trivial reflow biome itself proposes (no logic change) so this PR can land. Unrelated to DK-5.

That reflow also pushed run-review.mts (already at 640 lines, 17 over its recorded
eslint/baselines/size-lines.json ceiling of 623 from #94 itself) over the line-count ratchet.
Bumped that one entry to its true current count (642) — a surgical 1-line baseline update rather
than a full guard-size freeze (which re-walks the whole repo and could touch unrelated entries).

Summary by CodeRabbit

New Features
- Ship operations now provide the exact base commit to downstream audit checks.
- Audit checks can scope comparisons to that specific ship base when available.
Bug Fixes
- Improved audit accuracy by avoiding stale or incorrectly inferred comparison bases.
Tests
- Added coverage for base propagation across standard ships, re-ships, overlays, and self-hosted hooks.
- Improved test isolation for Git configuration.

…odetect (DK-5) ## DK-5 — `ship --base` audits against `main`, not the real ship base `devkit ship --base ` cuts the ephemeral gate worktree from ``'s origin tip, but the in-chain `fallow audit` (both the self-host advisory fragment and the overlay gate) always ran bare — `fallow audit`, no `--base` — so fallow auto-detected `main` as its comparison ref regardless of what the worktree was actually cut from. Shipping `--base feat/studio` from a finalized checkout, where `feat/studio` is itself a long-lived stack with its own pre-existing (not-yet-merged) dead-code / duplication findings vs `main`, misreported those as "new" and failed the audit. Fallow's own `audit` command already grandfathers correctly when given the right ref: its `gate: new-only` default runs the base ref too and marks inherited findings `introduced=false`. The fix is to thread the exact commit ship already resolves the worktree from — through as `DEVKIT_SHIP_BASE_SHA` (exported by `ship-branch.sh`/`reship.sh`, alongside the existing `DEVKIT_SHIP_MODE`) — so devkit's own fallow fragments pass `fallow audit --base "$DEVKIT_SHIP_BASE_SHA"` when it's set, falling back to fallow's own default on a plain `git commit`. This is unconditional (not just under `--base`): even the default ship case is more precise than `main`-autodetection for any branch that isn't a fresh cut off main. `.husky/pre-commit` (this repo's own self-hosted hook) is regenerated in the same commit — required, since `self-host.test.mts`'s parity test compares it byte-for-byte against the generator. A narrower, separate bug (`link-gate-configs.sh` symlinking `fallow-baselines/` from the invoking checkout rather than the ship base — relevant only to consumers using fallow's own self-installed git hook) is deliberately out of scope here, mirroring how DK-3 split an analogous baseline-follows- ROOT bug into its own PR. ## Also in this PR `.husky/pre-push` runs the full, untargeted test suite on every push, so any pre-existing failure blocks shipping anything. `overlay.test.mts`'s `workRepo()`-based tests call `applyInit` in-process (unlike the ship-branch/reship/reconcile suites, which spawn devkit as a subprocess and isolate git via an explicit per-call `env`), so its internal `execFileSync('git', …)` calls (e.g. `installHealAlias`'s `alias.ci` collision check) inherited whatever `process.env` already was — including a developer machine's own real `~/.gitconfig`. `overlay.mts` deliberately never clobbers an existing `git ci` alias, so a machine with one already set makes that check correctly skip installing devkit's self-heal alias, which then read as a false test failure. Isolated `GIT_CONFIG_GLOBAL`/`GIT_CONFIG_SYSTEM` to `/dev/null` for the file's duration (restored after), matching the isolation every sibling git-integration test file already does. Unrelated to DK-5 itself, but bundled in since it's what blocked the push. ## Verification - `bun run test:run -- cli/__tests__/ship-branch.test.mts cli/__tests__/reship.test.mts cli/__tests__/self-host.test.mts cli/__tests__/husky-block.test.mts cli/__tests__/overlay.test.mts` — 153 pass - `bun run typecheck` clean - `bun run lint` clean - `bun run lint:structure` clean - `git diff .husky/pre-commit` confirmed scoped to only the `# devkit:fallow-advisory` fragment ## Also bundled: a one-line formatting fix `origin/main` advanced (PR #94) between this branch's creation and shipping; `gate-engine/review/ run-review.mts`'s `savePasses(...)` call in that merge violates biome's line-length rule. Since `bun run lint` (`biome check .`) scans the whole repo, this now blocks any ship off `main` — bundled the trivial reflow biome itself proposes (no logic change) so this PR can land. Unrelated to DK-5. That reflow also pushed `run-review.mts` (already at 640 lines, 17 over its recorded `eslint/baselines/size-lines.json` ceiling of 623 from #94 itself) over the line-count ratchet. Bumped that one entry to its true current count (642) — a surgical 1-line baseline update rather than a full `guard-size freeze` (which re-walks the whole repo and could touch unrelated entries).

coderabbitai · 2026-07-17T07:58:34Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: aad30a85-6330-45cf-8c4b-39c12604eb31

📥 Commits

Reviewing files that changed from the base of the PR and between ef94e3c and 5c130a1.

📒 Files selected for processing (13)

.husky/pre-commit
cli/__tests__/husky-block.test.mts
cli/__tests__/overlay.test.mts
cli/__tests__/reship.test.mts
cli/__tests__/self-host.test.mts
cli/__tests__/ship-branch.test.mts
cli/lib/husky/husky-block.mts
cli/lib/husky/self-host.mts
cli/lib/ship/commit-with-gate-capture.sh
cli/lib/ship/reship.sh
cli/lib/ship/ship-branch.sh
eslint/baselines/size-lines.json
gate-engine/review/run-review.mts

📝 Walkthrough

Walkthrough

Changes

Fallow base scoping

Layer / File(s)	Summary
Export the resolved ship base `cli/lib/ship/...`, `cli/__tests__/ship-branch.test.mts`, `cli/__tests__/reship.test.mts`	Ship and reship flows export `DEVKIT_SHIP_BASE_SHA`; tests verify fetched and default base values.
Scope generated fallow gates `.husky/pre-commit`, `cli/lib/husky/...`, `cli/__tests__/husky-block.test.mts`, `cli/__tests__/self-host.test.mts`, `cli/__tests__/overlay.test.mts`	Fallow hooks conditionally pass `--base <sha>` while retaining existing gate behavior and coverage.

Review cache formatting

Layer / File(s)	Summary
Format pass-cache write `gate-engine/review/run-review.mts`, `eslint/baselines/size-lines.json`	The pass-cache object is reformatted without changing its fields, and the size baseline is updated.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ShipFlow
  participant GeneratedHook
  participant Fallow
  ShipFlow->>GeneratedHook: Export DEVKIT_SHIP_BASE_SHA
  GeneratedHook->>Fallow: Run fallow audit with optional --base SHA
  Fallow-->>GeneratedHook: Return audit status

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: scoping fallow audit to the ship's actual base instead of main autodetection.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/dk5-fallow-audit-base-scope4

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

…y-default (3-sample contract) (#101) * feat(sentry): registry guard + devkit-managed commit-msg hook, hard-by-default (3-sample contract) Closes the wiring gap behind benord-labs/frink#105: the sentry judge shipped as a bin with zero consumer wiring, so frink hand-wired the hook line (and the completeness commit-msg line before it). Hooks now come from devkit. ## What ships - **Registry**: `sentry` joins `GUARD_IDS` (offerable at init/upgrade; NOT recommended — needs a Sentry-using product). Wizard/help/doctor entries. - **Devkit-managed `.husky/commit-msg`**: new `cli/lib/husky/commit-msg-block.mts` generates a marker-wrapped block when `review` and/or `sentry` is selected — PATH preamble, `|| var=$?` tested-status plumbing (husky `sh -e` vs exit-2 fail-open), completeness + sentry invocations with block guidance, explicit `exit 0`. Idempotent install/removal wired through init/upgrade/clean/doctor (`--fix` handles missing AND drift); consumer content outside markers preserved; standalone mode emits `command -v`-guarded global-bin lines. Package + standalone only (self-host opts out — would newly gate devkit's own commits; overlay uses core.hooksPath). - **Hard-by-default**: `envBool('SENTRY_HARD') ?? true` (completeness precedent; env-only soften, no config key). Samples follow the frink Target's confidence contract: default 3 when hard, 1 when warn (`resolveSamples`, override wins). `effectiveHard` (from #99) still downgrades empty-staged-diff/amend to warn. - Decision note appended under the 2026-07-12 hard-by-default Target; README bins table updated. - Drive-by: committed `.husky/pre-commit` regenerated (+22 lines, the #96 commit-terminal fragment) — fixes the pre-existing self-host parity test red on main. ## Review-round fix (first ship attempt) The correctness reviewer correctly FAILED the first attempt: the worktree base predated #98 (DK-5), so the whole-file ship would have reverted the `DEVKIT_SHIP_BASE_SHA` fallow scoping. Rebased onto current main (post-#97/#98); the DK-5 overlay-fallow test block is restored alongside the new commit-msg tests (67/67). ## Ratchet bump (cross-PR race, user-approved pattern) `size-lines.json` husky-block.mts 524→528: #99 froze the cap at 524 while #98 (merged after) grew the file to 528 — main was over its own cap before this PR touched it (this PR's only husky-block.mts change is exporting PATH_SETUP, zero growth). ## Verified Touched suites green (131 + 67); typecheck clean; e2e: init with review+sentry → doctor reports both hook blocks OK; deselect/clean remove the block; upgrade offers sentry as opt-in. Only remaining suite red is the pre-existing overlay `git ci` alias test (red on pristine origin/main). ## Review-round fix (second attempt): completeness fragment now fails CLOSED on exit 3 (strict-ship judge outage), mirroring the pre-commit AI fragments — the reviewer correctly rejected the silent continue. ## Flagged, not done (follow-ups) - critique is the next half-migrated judge (devkit engine + eval exist, no CLI, frink runs local check-critique). - frink redo after next release + pin bump: select sentry guard, delete hand-wired lines + `check:sentry` script, rework #105. * fix(sentry-guard): review round — self-host block removal, deselected-fragment drift, test tmp cleanup CodeRabbit round: (1) package→self-host re-init now DROPS a previously-installed commit-msg block (stale hard gate would keep judging devkit's own commits); the doctor half of that finding is not applicable — self-host routes to runSelfHostDoctor before checkCommitMsgHook can run. (2) checkCommitMsgHook now flags DESELECTED lingering fragments as DRIFT (symmetric sentinel check, same depth as pre-commit's checkHusky; full-text equality would exceed the house standard). (3) test watchlist tmp dirs tracked + reclaimed in afterEach.

…odetect (DK-5) (#98) ## DK-5 — `ship --base` audits against `main`, not the real ship base `devkit ship --base ` cuts the ephemeral gate worktree from ``'s origin tip, but the in-chain `fallow audit` (both the self-host advisory fragment and the overlay gate) always ran bare — `fallow audit`, no `--base` — so fallow auto-detected `main` as its comparison ref regardless of what the worktree was actually cut from. Shipping `--base feat/studio` from a finalized checkout, where `feat/studio` is itself a long-lived stack with its own pre-existing (not-yet-merged) dead-code / duplication findings vs `main`, misreported those as "new" and failed the audit. Fallow's own `audit` command already grandfathers correctly when given the right ref: its `gate: new-only` default runs the base ref too and marks inherited findings `introduced=false`. The fix is to thread the exact commit ship already resolves the worktree from — through as `DEVKIT_SHIP_BASE_SHA` (exported by `ship-branch.sh`/`reship.sh`, alongside the existing `DEVKIT_SHIP_MODE`) — so devkit's own fallow fragments pass `fallow audit --base "$DEVKIT_SHIP_BASE_SHA"` when it's set, falling back to fallow's own default on a plain `git commit`. This is unconditional (not just under `--base`): even the default ship case is more precise than `main`-autodetection for any branch that isn't a fresh cut off main. `.husky/pre-commit` (this repo's own self-hosted hook) is regenerated in the same commit — required, since `self-host.test.mts`'s parity test compares it byte-for-byte against the generator. A narrower, separate bug (`link-gate-configs.sh` symlinking `fallow-baselines/` from the invoking checkout rather than the ship base — relevant only to consumers using fallow's own self-installed git hook) is deliberately out of scope here, mirroring how DK-3 split an analogous baseline-follows- ROOT bug into its own PR. ## Also in this PR `.husky/pre-push` runs the full, untargeted test suite on every push, so any pre-existing failure blocks shipping anything. `overlay.test.mts`'s `workRepo()`-based tests call `applyInit` in-process (unlike the ship-branch/reship/reconcile suites, which spawn devkit as a subprocess and isolate git via an explicit per-call `env`), so its internal `execFileSync('git', …)` calls (e.g. `installHealAlias`'s `alias.ci` collision check) inherited whatever `process.env` already was — including a developer machine's own real `~/.gitconfig`. `overlay.mts` deliberately never clobbers an existing `git ci` alias, so a machine with one already set makes that check correctly skip installing devkit's self-heal alias, which then read as a false test failure. Isolated `GIT_CONFIG_GLOBAL`/`GIT_CONFIG_SYSTEM` to `/dev/null` for the file's duration (restored after), matching the isolation every sibling git-integration test file already does. Unrelated to DK-5 itself, but bundled in since it's what blocked the push. ## Verification - `bun run test:run -- cli/__tests__/ship-branch.test.mts cli/__tests__/reship.test.mts cli/__tests__/self-host.test.mts cli/__tests__/husky-block.test.mts cli/__tests__/overlay.test.mts` — 153 pass - `bun run typecheck` clean - `bun run lint` clean - `bun run lint:structure` clean - `git diff .husky/pre-commit` confirmed scoped to only the `# devkit:fallow-advisory` fragment ## Also bundled: a one-line formatting fix `origin/main` advanced (PR #94) between this branch's creation and shipping; `gate-engine/review/ run-review.mts`'s `savePasses(...)` call in that merge violates biome's line-length rule. Since `bun run lint` (`biome check .`) scans the whole repo, this now blocks any ship off `main` — bundled the trivial reflow biome itself proposes (no logic change) so this PR can land. Unrelated to DK-5. That reflow also pushed `run-review.mts` (already at 640 lines, 17 over its recorded `eslint/baselines/size-lines.json` ceiling of 623 from #94 itself) over the line-count ratchet. Bumped that one entry to its true current count (642) — a surgical 1-line baseline update rather than a full `guard-size freeze` (which re-walks the whole repo and could touch unrelated entries).

…y-default (3-sample contract) (#101) * feat(sentry): registry guard + devkit-managed commit-msg hook, hard-by-default (3-sample contract) Closes the wiring gap behind benord-labs/frink#105: the sentry judge shipped as a bin with zero consumer wiring, so frink hand-wired the hook line (and the completeness commit-msg line before it). Hooks now come from devkit. ## What ships - **Registry**: `sentry` joins `GUARD_IDS` (offerable at init/upgrade; NOT recommended — needs a Sentry-using product). Wizard/help/doctor entries. - **Devkit-managed `.husky/commit-msg`**: new `cli/lib/husky/commit-msg-block.mts` generates a marker-wrapped block when `review` and/or `sentry` is selected — PATH preamble, `|| var=$?` tested-status plumbing (husky `sh -e` vs exit-2 fail-open), completeness + sentry invocations with block guidance, explicit `exit 0`. Idempotent install/removal wired through init/upgrade/clean/doctor (`--fix` handles missing AND drift); consumer content outside markers preserved; standalone mode emits `command -v`-guarded global-bin lines. Package + standalone only (self-host opts out — would newly gate devkit's own commits; overlay uses core.hooksPath). - **Hard-by-default**: `envBool('SENTRY_HARD') ?? true` (completeness precedent; env-only soften, no config key). Samples follow the frink Target's confidence contract: default 3 when hard, 1 when warn (`resolveSamples`, override wins). `effectiveHard` (from #99) still downgrades empty-staged-diff/amend to warn. - Decision note appended under the 2026-07-12 hard-by-default Target; README bins table updated. - Drive-by: committed `.husky/pre-commit` regenerated (+22 lines, the #96 commit-terminal fragment) — fixes the pre-existing self-host parity test red on main. ## Review-round fix (first ship attempt) The correctness reviewer correctly FAILED the first attempt: the worktree base predated #98 (DK-5), so the whole-file ship would have reverted the `DEVKIT_SHIP_BASE_SHA` fallow scoping. Rebased onto current main (post-#97/#98); the DK-5 overlay-fallow test block is restored alongside the new commit-msg tests (67/67). ## Ratchet bump (cross-PR race, user-approved pattern) `size-lines.json` husky-block.mts 524→528: #99 froze the cap at 524 while #98 (merged after) grew the file to 528 — main was over its own cap before this PR touched it (this PR's only husky-block.mts change is exporting PATH_SETUP, zero growth). ## Verified Touched suites green (131 + 67); typecheck clean; e2e: init with review+sentry → doctor reports both hook blocks OK; deselect/clean remove the block; upgrade offers sentry as opt-in. Only remaining suite red is the pre-existing overlay `git ci` alias test (red on pristine origin/main). ## Review-round fix (second attempt): completeness fragment now fails CLOSED on exit 3 (strict-ship judge outage), mirroring the pre-commit AI fragments — the reviewer correctly rejected the silent continue. ## Flagged, not done (follow-ups) - critique is the next half-migrated judge (devkit engine + eval exist, no CLI, frink runs local check-critique). - frink redo after next release + pin bump: select sentry guard, delete hand-wired lines + `check:sentry` script, rework #105. * fix(sentry-guard): review round — self-host block removal, deselected-fragment drift, test tmp cleanup CodeRabbit round: (1) package→self-host re-init now DROPS a previously-installed commit-msg block (stale hard gate would keep judging devkit's own commits); the doctor half of that finding is not applicable — self-host routes to runSelfHostDoctor before checkCommitMsgHook can run. (2) checkCommitMsgHook now flags DESELECTED lingering fragments as DRIFT (symmetric sentinel check, same depth as pre-commit's checkHusky; full-text equality would exceed the house standard). (3) test watchlist tmp dirs tracked + reclaimed in afterEach.

norvalbv merged commit deef7e0 into main Jul 17, 2026
1 check passed

norvalbv mentioned this pull request Jul 17, 2026

feat(sentry): registry guard + devkit-managed commit-msg hook, hard-by-default (3-sample contract) #101

Merged

This was referenced Jul 17, 2026

feat: add devkit review workflow #95

Closed

fix(ship): pin fallow >=3.6.0 — its audit worktree cleanup clobbered the ship worktree's HEAD #122

Merged

feat: configure review-mode gate routing #108

Merged

This was referenced Aug 6, 2026

fix(ship): skip the redundant pre-push suite for devkit ship pushes (sc-1508) #354

Merged

fix(guard-size): scope pull-request CI to the PR diff (sc-1539) #378

Merged

Navigation Menu

fix(ship): scope fallow audit to the ship's actual base, not main-autodetect (DK-5) - #98

fix(ship): scope fallow audit to the ship's actual base, not main-autodetect (DK-5)#98
norvalbv merged 1 commit into
mainfrom
fix/dk5-fallow-audit-base-scope4

norvalbv commented Jul 17, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 17, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

norvalbv commented Jul 17, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

DK-5 — ship --base audits against main, not the real ship base

Also in this PR

Verification

Also bundled: a one-line formatting fix

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

norvalbv commented Jul 17, 2026 •

edited by coderabbitai Bot

Loading

DK-5 — `ship --base` audits against `main`, not the real ship base

coderabbitai Bot commented Jul 17, 2026 •

edited

Loading