status: publish stale_pending, the mark that predicts a slow get - #19
Conversation
credential.status reported ready:true / last_error_code:null on a record whose next get was about to buy an upstream token exchange -- measured 2026-08-25, twelve samples over five minutes with the chain row already written. `ready` answers "would a get succeed" (genuinely yes; the mark exists so the next get refreshes rather than refusing) and could not answer what that get would cost: active, stale_pending=false -> local read, sub-millisecond active, stale_pending=TRUE -> forces an upstream exchange, seconds needs_reauth -> fails fast, no upstream call Withheld until now on the grounds that a field added for a hypothetical caller is machinery nobody can test against a real requirement. That condition ended: the first vault consumer warms a credential cache at startup and needs to SKIP the accounts that would overrun its bound rather than discover them by timing out -- and the only other way to observe the mark was credential.get, the very call whose cost is in question. Free to serve: stale_pending is a plaintext column, store::meta() already selects it, and RecordMeta already carries it. This path was fetching the value and discarding it before serialization -- the state record_version was in before it was published. No decrypt, no master key, no extra query. Absent rather than false when no handle resolved: a defaulted false would read as "no repair pending" for a revoked handle, an assertion this path cannot make. Test pins both halves and was mutation-checked (publish -> None turns it red): the mark is visible without calling get, AND ready stays true -- expensive is not unhealthy. If that second assertion ever flips, the field has been folded into health and consumers will start treating usable credentials as broken. Lockfile regenerated against subconscious f52c1309 (subc-core 0.9.0->0.11.0, subc-control 0.7.0->0.9.0). Wire crates unchanged: subc-protocol 0.13.0, subc-transport 0.5.1 -- protocol 2 compatibility preserved. Gate: 421 tests, exit 0.
There was a problem hiding this comment.
2 issues found across 2 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="crates/credentials-module/src/read_surface.rs">
<violation number="1" location="crates/credentials-module/src/read_surface.rs:1240">
P2: When a marked credential is later invalidated, retired, or quarantined, `status` reports `stale_pending: true` even though the next `get` fails fast and cannot perform the predicted refresh. Publish the marker only for `Active` records, or clear it whenever the record leaves `Active`.</violation>
</file>
<file name="crates/credentials-module/src/main.rs">
<violation number="1" location="crates/credentials-module/src/main.rs:4175">
P2: This test does not exercise the slow-refresh case it describes: `apikey:active` is non-refreshable, and the public report path never sets `stale_pending` for it. Use a refreshable OAuth fixture and exercise `report_auth_failure` so the test verifies the field for the production path that can incur an upstream exchange.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| && matches!(meta.state, credentials_core::store::RecordState::Active), | ||
| // Deliberately NOT folded into `ready`: a stale-marked record is | ||
| // still usable, it is merely expensive on the next read. | ||
| stale_pending: Some(meta.stale_pending), |
There was a problem hiding this comment.
P2: When a marked credential is later invalidated, retired, or quarantined, status reports stale_pending: true even though the next get fails fast and cannot perform the predicted refresh. Publish the marker only for Active records, or clear it whenever the record leaves Active.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/credentials-module/src/read_surface.rs, line 1240:
<comment>When a marked credential is later invalidated, retired, or quarantined, `status` reports `stale_pending: true` even though the next `get` fails fast and cannot perform the predicted refresh. Publish the marker only for `Active` records, or clear it whenever the record leaves `Active`.</comment>
<file context>
@@ -1202,6 +1235,9 @@ impl ReadSurface {
&& matches!(meta.state, credentials_core::store::RecordState::Active),
+ // Deliberately NOT folded into `ready`: a stale-marked record is
+ // still usable, it is merely expensive on the next read.
+ stale_pending: Some(meta.stale_pending),
last_error_code: match meta.state {
credentials_core::store::RecordState::NeedsReauth
</file context>
| stale_pending: Some(meta.stale_pending), | |
| stale_pending: matches!( | |
| meta.state, | |
| credentials_core::store::RecordState::Active, | |
| ) | |
| .then_some(meta.stale_pending), |
| store | ||
| .put_handle_hash( | ||
| &handle.hash, | ||
| "apikey:active", |
There was a problem hiding this comment.
P2: This test does not exercise the slow-refresh case it describes: apikey:active is non-refreshable, and the public report path never sets stale_pending for it. Use a refreshable OAuth fixture and exercise report_auth_failure so the test verifies the field for the production path that can incur an upstream exchange.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/credentials-module/src/main.rs, line 4175:
<comment>This test does not exercise the slow-refresh case it describes: `apikey:active` is non-refreshable, and the public report path never sets `stale_pending` for it. Use a refreshable OAuth fixture and exercise `report_auth_failure` so the test verifies the field for the production path that can incur an upstream exchange.</comment>
<file context>
@@ -4158,6 +4158,105 @@ mod tests {
+ store
+ .put_handle_hash(
+ &handle.hash,
+ "apikey:active",
+ AuditCtx::admin(AuditOp::MintHandle),
+ )
</file context>
The existing golden test pins the four `class` wire strings against the contract's canonical set. It says nothing about the envelope they arrive in. Rename `class` to `error_class`, move the error a nesting level, or drop the field from the body, and that test stays green while every consumer breaks. Found by the first live consumer of this surface. They typed a decoder from the published contract, parsed a real error frame SUCCESSFULLY, and discarded `class` entirely -- serde drops unknown fields without complaint, so a decoder ignoring the field it was told to branch on is indistinguishable from one honouring it. They then branched on `code` through a closed enum with no fallback, which turns the first added code into a parse failure rather than an unknown-code branch, and their untagged wrapper reported it as a malformed local config file. None of that is fixable from this side. What is fixable is guaranteeing the bytes never move under them, so a consumer that gets it right stays right. Serialized through the real producer type rather than a hand-built json!, so it pins what the wire carries; a reconstruction would only pin the reconstruction. The literal is the exact frame captured from a live daemon and handed to that consumer, who pinned the same bytes on their side -- drift now goes red in both directions, which neither test can achieve alone. Assertion order is load-bearing and this is the second version. Written with the equality first, the specific check never ran: assert_eq! panics on any difference, so dropping `class` reported "the frame shape drifted" and left the reader to diff two blobs -- the diagnostic existed only for the case it could not reach. A cheap specific assertion must precede a broad one that subsumes it, or it is decoration. Mutation-checked both ways: renaming the field and skipping it each turn it red with the specific message, and restoring returns it green. Gate: 421 tests, exit 0, nine real-daemon e2e arms executing.
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="crates/credentials-module/src/read_surface.rs">
<violation number="1" location="crates/credentials-module/src/read_surface.rs:1523">
P3: The comment says the pinned literal is "the exact frame captured from a live daemon", but the pin covers only the inner error body. Every transport path (e.g. `json!({ "result": ... })` in main.rs) wraps it under a `result` key, so a change to that wrapper (renaming `result`/`error` at the response level) leaves this test green while consuming decoders break. Say the pin is scoped to the error-body shape, not the full on-wire frame.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| /// this pins what the wire actually carries. A reconstruction would only pin the | ||
| /// reconstruction — the frame could drift and this would still pass. | ||
| /// | ||
| /// The literal is the exact frame captured from a live daemon and handed to that |
There was a problem hiding this comment.
P3: The comment says the pinned literal is "the exact frame captured from a live daemon", but the pin covers only the inner error body. Every transport path (e.g. json!({ "result": ... }) in main.rs) wraps it under a result key, so a change to that wrapper (renaming result/error at the response level) leaves this test green while consuming decoders break. Say the pin is scoped to the error-body shape, not the full on-wire frame.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/credentials-module/src/read_surface.rs, line 1523:
<comment>The comment says the pinned literal is "the exact frame captured from a live daemon", but the pin covers only the inner error body. Every transport path (e.g. `json!({ "result": ... })` in main.rs) wraps it under a `result` key, so a change to that wrapper (renaming `result`/`error` at the response level) leaves this test green while consuming decoders break. Say the pin is scoped to the error-body shape, not the full on-wire frame.</comment>
<file context>
@@ -1502,6 +1502,59 @@ mod error_class_tests {
+ /// this pins what the wire actually carries. A reconstruction would only pin the
+ /// reconstruction — the frame could drift and this would still pass.
+ ///
+ /// The literal is the exact frame captured from a live daemon and handed to that
+ /// consumer, who pinned it in their tree. Both directions now go red on drift.
+ #[test]
</file context>
| /// The literal is the exact frame captured from a live daemon and handed to that | |
| /// The literal is the exact error-body frame captured from a live daemon and handed to that |
credential.statusreportedready: true/last_error_code: nullon a record whose nextgetwas about to buy an upstream token exchange. Twelve consecutive samples across a five-minute window, with theconsumer_report_stalechain row already written.readywas not wrong. It answers "would a get succeed" — and it would, because the mark exists precisely so the nextgetrefreshes rather than refusing. It could not answer what thatgetwould cost:The middle row reads as healthy and is about to be expensive. On a deployment whose upstream credential rotates, it is a ~300s window every four hours.
Why now, when it was deliberately withheld
The read surface carried a note that a field added for a hypothetical caller is machinery nobody can test against a real requirement, and to revisit when one arrived. One has. A consumer warms a credential cache at startup behind a bounded await and needs to skip the accounts that would overrun the bound rather than discover them by timing out. The only other way to observe the mark was
credential.get— the call whose cost is in question.Free to serve
stale_pendingis a plaintext column.store::meta()already selects it andRecordMetaalready carries it; this path was fetching the value and discarding it before serialization. No decrypt, no master key, no additional query. It is the staterecord_versionwas in before it was published.Absent, not false, when no handle resolved
{"result":{"last_error_code":null,"ready":true,"record_version":65,"stale_pending":false}} {"result":{"last_error_code":"not_found","ready":false}}A defaulted
falseon the second would assert "no repair pending" about a credential the path cannot see. Consumers act on explicittrueonly, so absence degrades to "check it the slow way" rather than to a confident wrong answer.Test
Pins both halves, and the second is load-bearing: the mark is visible without calling
get, andreadystaystruewhile it is set. Expensive is not unhealthy. If that assertion ever flips, the field has been folded into health and consumers will begin treating usable credentials as broken.Mutation-checked — publishing
Nonein place of the value reddens it.statusstill writes nothing to the chain: it resolves the handle and reads plaintext metadata, never reachingengine.get(), so polling it cannot trigger the refresh it predicts. Chain tip held constant across every probe during verification. It does still count against the fetch limiter, which is worth knowing before anyone polls it per request.Verification
bash scripts/gate.shgreen, exit 0, with the nine real-daemon e2e arms executing rather than skipped. Both response shapes above were read off the wire from a live daemon running this commit.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Previously
credential.statusreportedready: trueeven when the nextgetwould trigger an upstream token exchange. It now publishesstale_pending— a latency predictor, not a health field — so consumers can see the pending repair before paying that cost, and it pins the error frame's wire shape so the contract can't drift unnoticed.Notes
false.statusnever triggers the refresh it predicts, but it does count against the fetch limiter.store::meta()already selects andRecordMetaalready carries.Error frame pin
classstrings, so renaming, moving, or dropping the key could pass while consumers broke.Written for commit 8d3d81c. Summary will update on new commits.