fix(qavis): record outcomes and attribute strict ship blocks [sc-2570] - #606
Merged
Conversation
Qavis advisory telemetry previously recorded bypasses but omitted ordinary clearance, unmet QA requirements and self-run outcomes. That made Frink's recorded Qavis events look entirely bypassed and prevented measuring which changes actually needed QA. Emit a terminal outcome and duration for SILENT, required QA, unavailable routing and self-run rechecks. Preserve the existing contract: valid unmet requirements block strict ships; ordinary commits remain advisory; unavailable classification fails open visibly. A SILENT gate result is gate clearance, not a fabricated visual-test pass. Include the self-run exit code and keep bypass flag events distinct. Strict Qavis blocks are attributed in the terminal ship summary, and unavailable-classifier reasons survive into its digest. Compiled consumer artifacts are included. Validation: typecheck/build, 86 Qavis advisory, ship-capture and digest tests, staged anti-slop and normal commit gates. Tests cover strict/advisory results, all self-run rechecks, unavailable classification and a broken telemetry sink. Fallow reports the existing branch-heavy advisory function; the branching is unchanged. Clone checking and the normal commit reviewers passed. Completes the telemetry portion of [sc-2570](https://app.shortcut.com/benordlabs/story/2570). Companion [Qavis PR #105](benord-labs/qavis#105) rejects malformed classifier verdicts instead of inventing ADVISE, skips universally inert deltas, and speeds transport/CLI loading. No production bypass-rate claim is possible until normal outcome events accumulate. Full local pre-push validation: 6,734 tests passed; one reship receipt-isolation test hit its 150s timeout during the 59-minute run. The exact test passed alone in 25.73s. Publication uses the standard devkit ship workflow; PR CI remains responsible for its full-suite result. No assertion or timeout threshold was changed.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: ⛔ Files ignored due to path filters (3)
📒 Files selected for processing (8)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughQavis gate execution now emits structured outcome and timing telemetry, records self-run exit codes, preserves unavailable-route details, and marks strict blocks in ship results. Tests cover these paths and update ship attribution expectations. ChangesQavis telemetry and attribution
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to Qavis now records terminal gate outcomes, timings, routing details, self-run exit codes, and strict-block ship attribution while retaining existing strict and advisory behavior. The covered paths show no current merge-blocking risk. Sequence Diagram(s)sequenceDiagram
participant ShipCommand
participant runQavisAdvisory
participant TelemetrySink
participant ShipDigest
ShipCommand->>runQavisAdvisory: run Qavis gate
runQavisAdvisory->>TelemetrySink: emit outcome and timing events
runQavisAdvisory-->>ShipCommand: return gate result
ShipCommand->>ShipDigest: capture strict Qavis marker
ShipDigest-->>ShipCommand: record blocked_gate=qavis-advisory
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 6 files. (2 skipped: 2 unsupported.)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Qavis advisory telemetry previously recorded bypasses but omitted ordinary clearance, unmet QA requirements and self-run outcomes. That made Frink's recorded Qavis events look entirely bypassed and prevented measuring which changes actually needed QA.
Emit a terminal outcome and duration for SILENT, required QA, unavailable routing and self-run rechecks. Preserve the existing contract: valid unmet requirements block strict ships; ordinary commits remain advisory; unavailable classification fails open visibly. A SILENT gate result is gate clearance, not a fabricated visual-test pass. Include the self-run exit code and keep bypass flag events distinct. Strict Qavis blocks are attributed in the terminal ship summary, and unavailable-classifier reasons survive into its digest. Compiled consumer artifacts are included.
Validation: typecheck/build, 86 Qavis advisory, ship-capture and digest tests, staged anti-slop and normal commit gates. Tests cover strict/advisory results, all self-run rechecks, unavailable classification and a broken telemetry sink. Fallow reports the existing branch-heavy advisory function; the branching is unchanged. Clone checking and the normal commit reviewers passed.
Completes the telemetry portion of sc-2570. Companion Qavis PR #105 rejects malformed classifier verdicts instead of inventing ADVISE, skips universally inert deltas, and speeds transport/CLI loading. No production bypass-rate claim is possible until normal outcome events accumulate.
Full local pre-push validation: 6,734 tests passed; one reship receipt-isolation test hit its 150s timeout during the 59-minute run. The exact test passed alone in 25.73s. Publication uses the standard devkit ship workflow; PR CI remains responsible for its full-suite result. No assertion or timeout threshold was changed.
Summary by CodeRabbit
New Features
Documentation
Tests