Skip to content

status: publish stale_pending, the mark that predicts a slow get - #19

Merged
ualtinok merged 2 commits into
cortexkit:masterfrom
legion-works:feat/status-stale-pending
Aug 29, 2026
Merged

status: publish stale_pending, the mark that predicts a slow get#19
ualtinok merged 2 commits into
cortexkit:masterfrom
legion-works:feat/status-stale-pending

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

credential.status reported ready: true / last_error_code: null on a record whose next get was about to buy an upstream token exchange. Twelve consecutive samples across a five-minute window, with the consumer_report_stale chain row already written.

ready was not wrong. It answers "would a get succeed" — and it would, because the mark exists precisely so the next get refreshes rather than refusing. It could not answer what that get would cost:

active, stale_pending = false   local read, sub-millisecond
active, stale_pending = TRUE    forces an upstream exchange, seconds
needs_reauth                    fails fast, no upstream call

The middle row reads as healthy and is about to be expensive. On a deployment whose upstream credential rotates, it is a ~300s window every four hours.

Why now, when it was deliberately withheld

The read surface carried a note that a field added for a hypothetical caller is machinery nobody can test against a real requirement, and to revisit when one arrived. One has. A consumer warms a credential cache at startup behind a bounded await and needs to skip the accounts that would overrun the bound rather than discover them by timing out. The only other way to observe the mark was credential.get — the call whose cost is in question.

Free to serve

stale_pending is a plaintext column. store::meta() already selects it and RecordMeta already carries it; this path was fetching the value and discarding it before serialization. No decrypt, no master key, no additional query. It is the state record_version was in before it was published.

Absent, not false, when no handle resolved

{"result":{"last_error_code":null,"ready":true,"record_version":65,"stale_pending":false}}
{"result":{"last_error_code":"not_found","ready":false}}

A defaulted false on the second would assert "no repair pending" about a credential the path cannot see. Consumers act on explicit true only, so absence degrades to "check it the slow way" rather than to a confident wrong answer.

Test

Pins both halves, and the second is load-bearing: the mark is visible without calling get, and ready stays true while it is set. Expensive is not unhealthy. If that assertion ever flips, the field has been folded into health and consumers will begin treating usable credentials as broken.

Mutation-checked — publishing None in place of the value reddens it.

status still writes nothing to the chain: it resolves the handle and reads plaintext metadata, never reaching engine.get(), so polling it cannot trigger the refresh it predicts. Chain tip held constant across every probe during verification. It does still count against the fetch limiter, which is worth knowing before anyone polls it per request.

Verification

bash scripts/gate.sh green, exit 0, with the nine real-daemon e2e arms executing rather than skipped. Both response shapes above were read off the wire from a live daemon running this commit.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Previously credential.status reported ready: true even when the next get would trigger an upstream token exchange. It now publishes stale_pending — a latency predictor, not a health field — so consumers can see the pending repair before paying that cost, and it pins the error frame's wire shape so the contract can't drift unnoticed.

Notes

  • The field is present only when a handle resolves; unresolvable handles omit it rather than defaulting to false.
  • A consumer warming a credential cache can now skip accounts that would overrun its startup bound instead of timing out.
  • status never triggers the refresh it predicts, but it does count against the fetch limiter.
  • The flag is free to serve: a plaintext column store::meta() already selects and RecordMeta already carries.

Error frame pin

  • The existing test only pinned the four class strings, so renaming, moving, or dropping the key could pass while consumers broke.
  • The new test serializes through the real producer type and matches the exact frame captured from a live daemon.

Written for commit 8d3d81c. Summary will update on new commits.

Review in cubic

credential.status reported ready:true / last_error_code:null on a record whose
next get was about to buy an upstream token exchange -- measured 2026-08-25,
twelve samples over five minutes with the chain row already written. `ready`
answers "would a get succeed" (genuinely yes; the mark exists so the next get
refreshes rather than refusing) and could not answer what that get would cost:

  active, stale_pending=false  -> local read, sub-millisecond
  active, stale_pending=TRUE   -> forces an upstream exchange, seconds
  needs_reauth                 -> fails fast, no upstream call

Withheld until now on the grounds that a field added for a hypothetical caller
is machinery nobody can test against a real requirement. That condition ended:
the first vault consumer warms a credential cache at startup and needs to SKIP
the accounts that would overrun its bound rather than discover them by timing
out -- and the only other way to observe the mark was credential.get, the very
call whose cost is in question.

Free to serve: stale_pending is a plaintext column, store::meta() already
selects it, and RecordMeta already carries it. This path was fetching the value
and discarding it before serialization -- the state record_version was in before
it was published. No decrypt, no master key, no extra query.

Absent rather than false when no handle resolved: a defaulted false would read
as "no repair pending" for a revoked handle, an assertion this path cannot make.

Test pins both halves and was mutation-checked (publish -> None turns it red):
the mark is visible without calling get, AND ready stays true -- expensive is
not unhealthy. If that second assertion ever flips, the field has been folded
into health and consumers will start treating usable credentials as broken.

Lockfile regenerated against subconscious f52c1309 (subc-core 0.9.0->0.11.0,
subc-control 0.7.0->0.9.0). Wire crates unchanged: subc-protocol 0.13.0,
subc-transport 0.5.1 -- protocol 2 compatibility preserved.

Gate: 421 tests, exit 0.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/credentials-module/src/read_surface.rs">

<violation number="1" location="crates/credentials-module/src/read_surface.rs:1240">
P2: When a marked credential is later invalidated, retired, or quarantined, `status` reports `stale_pending: true` even though the next `get` fails fast and cannot perform the predicted refresh. Publish the marker only for `Active` records, or clear it whenever the record leaves `Active`.</violation>
</file>

<file name="crates/credentials-module/src/main.rs">

<violation number="1" location="crates/credentials-module/src/main.rs:4175">
P2: This test does not exercise the slow-refresh case it describes: `apikey:active` is non-refreshable, and the public report path never sets `stale_pending` for it. Use a refreshable OAuth fixture and exercise `report_auth_failure` so the test verifies the field for the production path that can incur an upstream exchange.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

&& matches!(meta.state, credentials_core::store::RecordState::Active),
// Deliberately NOT folded into `ready`: a stale-marked record is
// still usable, it is merely expensive on the next read.
stale_pending: Some(meta.stale_pending),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When a marked credential is later invalidated, retired, or quarantined, status reports stale_pending: true even though the next get fails fast and cannot perform the predicted refresh. Publish the marker only for Active records, or clear it whenever the record leaves Active.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/credentials-module/src/read_surface.rs, line 1240:

<comment>When a marked credential is later invalidated, retired, or quarantined, `status` reports `stale_pending: true` even though the next `get` fails fast and cannot perform the predicted refresh. Publish the marker only for `Active` records, or clear it whenever the record leaves `Active`.</comment>

<file context>
@@ -1202,6 +1235,9 @@ impl ReadSurface {
                         && matches!(meta.state, credentials_core::store::RecordState::Active),
+                    // Deliberately NOT folded into `ready`: a stale-marked record is
+                    // still usable, it is merely expensive on the next read.
+                    stale_pending: Some(meta.stale_pending),
                     last_error_code: match meta.state {
                         credentials_core::store::RecordState::NeedsReauth
</file context>
Suggested change
stale_pending: Some(meta.stale_pending),
stale_pending: matches!(
meta.state,
credentials_core::store::RecordState::Active,
)
.then_some(meta.stale_pending),

store
.put_handle_hash(
&handle.hash,
"apikey:active",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This test does not exercise the slow-refresh case it describes: apikey:active is non-refreshable, and the public report path never sets stale_pending for it. Use a refreshable OAuth fixture and exercise report_auth_failure so the test verifies the field for the production path that can incur an upstream exchange.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/credentials-module/src/main.rs, line 4175:

<comment>This test does not exercise the slow-refresh case it describes: `apikey:active` is non-refreshable, and the public report path never sets `stale_pending` for it. Use a refreshable OAuth fixture and exercise `report_auth_failure` so the test verifies the field for the production path that can incur an upstream exchange.</comment>

<file context>
@@ -4158,6 +4158,105 @@ mod tests {
+        store
+            .put_handle_hash(
+                &handle.hash,
+                "apikey:active",
+                AuditCtx::admin(AuditOp::MintHandle),
+            )
</file context>

The existing golden test pins the four `class` wire strings against the
contract's canonical set. It says nothing about the envelope they arrive in.
Rename `class` to `error_class`, move the error a nesting level, or drop the
field from the body, and that test stays green while every consumer breaks.

Found by the first live consumer of this surface. They typed a decoder from the
published contract, parsed a real error frame SUCCESSFULLY, and discarded
`class` entirely -- serde drops unknown fields without complaint, so a decoder
ignoring the field it was told to branch on is indistinguishable from one
honouring it. They then branched on `code` through a closed enum with no
fallback, which turns the first added code into a parse failure rather than an
unknown-code branch, and their untagged wrapper reported it as a malformed
local config file.

None of that is fixable from this side. What is fixable is guaranteeing the
bytes never move under them, so a consumer that gets it right stays right.

Serialized through the real producer type rather than a hand-built json!, so it
pins what the wire carries; a reconstruction would only pin the reconstruction.
The literal is the exact frame captured from a live daemon and handed to that
consumer, who pinned the same bytes on their side -- drift now goes red in both
directions, which neither test can achieve alone.

Assertion order is load-bearing and this is the second version. Written with
the equality first, the specific check never ran: assert_eq! panics on any
difference, so dropping `class` reported "the frame shape drifted" and left the
reader to diff two blobs -- the diagnostic existed only for the case it could
not reach. A cheap specific assertion must precede a broad one that subsumes
it, or it is decoration.

Mutation-checked both ways: renaming the field and skipping it each turn it red
with the specific message, and restoring returns it green.

Gate: 421 tests, exit 0, nine real-daemon e2e arms executing.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/credentials-module/src/read_surface.rs">

<violation number="1" location="crates/credentials-module/src/read_surface.rs:1523">
P3: The comment says the pinned literal is "the exact frame captured from a live daemon", but the pin covers only the inner error body. Every transport path (e.g. `json!({ "result": ... })` in main.rs) wraps it under a `result` key, so a change to that wrapper (renaming `result`/`error` at the response level) leaves this test green while consuming decoders break. Say the pin is scoped to the error-body shape, not the full on-wire frame.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

/// this pins what the wire actually carries. A reconstruction would only pin the
/// reconstruction — the frame could drift and this would still pass.
///
/// The literal is the exact frame captured from a live daemon and handed to that

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The comment says the pinned literal is "the exact frame captured from a live daemon", but the pin covers only the inner error body. Every transport path (e.g. json!({ "result": ... }) in main.rs) wraps it under a result key, so a change to that wrapper (renaming result/error at the response level) leaves this test green while consuming decoders break. Say the pin is scoped to the error-body shape, not the full on-wire frame.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/credentials-module/src/read_surface.rs, line 1523:

<comment>The comment says the pinned literal is "the exact frame captured from a live daemon", but the pin covers only the inner error body. Every transport path (e.g. `json!({ "result": ... })` in main.rs) wraps it under a `result` key, so a change to that wrapper (renaming `result`/`error` at the response level) leaves this test green while consuming decoders break. Say the pin is scoped to the error-body shape, not the full on-wire frame.</comment>

<file context>
@@ -1502,6 +1502,59 @@ mod error_class_tests {
+    /// this pins what the wire actually carries. A reconstruction would only pin the
+    /// reconstruction — the frame could drift and this would still pass.
+    ///
+    /// The literal is the exact frame captured from a live daemon and handed to that
+    /// consumer, who pinned it in their tree. Both directions now go red on drift.
+    #[test]
</file context>
Suggested change
/// The literal is the exact frame captured from a live daemon and handed to that
/// The literal is the exact error-body frame captured from a live daemon and handed to that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants