Address review: the accounting edges, the report gate, and two dead controls

README.md

@@ -25,15 +25,15 @@ is a sentence in `prompts/system.md`.
 | A prediction exists before the result does | `core/gates.py:check_expectation`, bound at submit time |
 | Results get recorded at all | `collect` writes the run record; a stale uncollected run blocks new submissions |
 | Cumulative spend stays bounded | `core/ledger_store.py:rolling_spend` — actuals for collected runs, estimates for in-flight ones |
-| The smoke job cannot become a backdoor | `core/gates.py:check_smoke_caps` clamps steps, wall clock, and cost in code |
+| The smoke job cannot become a backdoor | `core/gates.py:check_smoke_caps` clamps steps and wall clock, and clamps the wall clock again against the target's hourly rate so the cost cap is arithmetic rather than a self-report |
 | Notebooks run clean top-to-bottom | `tools/nb.py verify` on a fresh kernel |
 | No general remote-execution capability | credentials in Windows Credential Manager, read only by `jobs.py` / `gpu.py` |
 | Concurrent ledger writes don't corrupt | one locked `core/jsonl.py:append`; no CLI writes a ledger file directly |
 | Token and credit spend stays bounded, not merely measured | `core/budget.py`, checked at every gateable event |
 | An evolutionary campaign cannot outspend its allocation | the campaign gate in `tools/evolve.py`, before generation 0 and before each generation after it |
 | A job submitted to an org is collectable from that org | the namespace is persisted on the run handle, not just passed at submit |
-| Every number in a report traces to a run record | `tools/report.py check` refuses on an unresolved claim |
-| Every citation in a report is a real paper | `report cite` resolves only against the corpus and verified S2 ids |
+| Every number in a report traces to a run record | `tools/report.py check` refuses on an unresolved claim, on a `claims.tex` that has drifted from `claims.json`, and on a measured-looking number typed into the generated prose |
+| Every citation in a report is a real paper | `report cite` resolves only against the corpus and verified S2 ids, and `check` re-resolves each entry's id rather than trusting its `gradsource` label |
 | A result that has not been judged cannot be published | `report check` refuses while any cited run has an unjudged deviation |
 
 ### The one thing that is *not* fully mechanical, and why
@@ -44,6 +44,13 @@ mid-turn, so `agent.py` checks the remaining allocation *before* issuing the
 next turn and `hooks.py` denies cost-bearing Bash once the project is over. The
 turn that crosses the ceiling finishes.
 
+Both surfaces run the same check because both run the same loop:
+`agent.drive_turn` is the one place a turn is issued, and it checks the budget
+before `query` and records the turn's usage after it. The CLI and the desktop
+app called it independently for a while, and only the CLI accounted — so a
+session held entirely in the app spent tokens that no ledger recorded and no
+ceiling could see.
+
 A second honesty note: subscription quota is not linear in tokens, and the real
 limits are rolling windows (5-hour and weekly on Max) that the SDK does not
 expose as a remaining balance. **A token ceiling is a proxy you control, not a
@@ -204,7 +211,7 @@ what each check catches.
 
 ```
 agent.py              ClaudeSDKClient loop, permission configuration, the deny probe
-hooks.py              PreToolUse gate (a speed bump) + Stop hook (quota accounting)
+hooks.py              PreToolUse gate (a speed bump) + Stop hook (budget warnings)
 prompts/system.md     under 1000 tokens
 core/                 the machinery the CLIs share, so no tool can forget a rule
   cli.py              the §8 CLI contract, implemented once
@@ -294,7 +301,7 @@ that does not import can only guess at what is installed. Run it on your own
 pipeline, not on a repository you just downloaded; the module docstring and
 `--help` both say so.
 
-Two things worth knowing before trusting them:
+Four things worth knowing before trusting them:
 
 - **The Agent SDK surface is version-sensitive.** `core/haiku.py` and
   `agent.py` are written against the interfaces described in the handoff
@@ -305,6 +312,21 @@ Two things worth knowing before trusting them:
   of one call, because `ssh` needs a key file. That is weaker than never
   materialising it. Prefer an SSH agent or a `~/.ssh/config` host entry and
   leave `key_credential` unset, in which case no key is ever written by us.
+- **The preflight record is a plain JSON file the agent can write.** Gate 1
+  reads `ledger/preflight/<hash>.json` and the model has `Write`. So the
+  cheapest way past the most important gate is not an argument, it is a file —
+  which puts it in the same class as the bypasses `core/credentials.py` already
+  declares out of scope (an agent that can run Python can import `keyring`).
+  Signing the record would not close it either, since the signing key would be
+  readable by the same process. What actually bounds this is that the *spend*
+  gates do not read agent-writable state: the ledger is append-only through one
+  locked path, and `collect` prices runs from the platform's own timestamps.
+- **The S2 half of the citation rule is weaker than the corpus half.** A
+  `corpus` entry is verified by resolving its document id against the local
+  index. An `s2` entry is verified by its `S2:<id>` shape and the overlap scores
+  `report cite` recorded when it accepted the match — re-querying the live
+  service inside a gate would make `check` require the network. Forging one is
+  no longer a single line of BibTeX, but it is not impossible.
 
 The order in §12 of the handoff is deliberate — build the agent, use it for a
 week, *then* harvest `evals/retrieval.jsonl` from what retrieval was actually

agent.py

@@ -170,7 +170,15 @@ def check_turn_budget() -> dict[str, Any] | None:
         if not project_id or not budget.exists(project_id):
             return None
         state = budget.status(project_id)
-    except Exception:  # noqa: BLE001 - accounting must never strand a session
+    except Exception as exc:  # noqa: BLE001 - accounting must never strand a session
+        # Fails open, and says so. This is the *only* mechanism that bounds
+        # token spend before a turn; if it cannot read the ledger, the honest
+        # report is that the turn is going out ungated.
+        print(
+            f"[grad] token budget check failed ({type(exc).__name__}: {exc}); "
+            "this turn is not gated",
+            file=sys.stderr,
+        )
         return None
 
     tokens = state["resources"]["quota_tokens"]
@@ -196,27 +204,93 @@ def check_turn_budget() -> dict[str, Any] | None:
     }
 
 
-async def _turn(client: Any, prompt: str) -> bool:
-    """Run one turn. Returns False if the budget refused it."""
+class BudgetRefused(Exception):
+    """Raised by `drive_turn` when the project is out of token allocation.
+
+    Carries the payload so a caller can render it: the CLI prints it, the UI
+    puts it in the transcript.
+    """
+
+    def __init__(self, refusal: dict[str, Any]) -> None:
+        super().__init__(refusal["message"])
+        self.refusal = refusal
+
+
+async def drive_turn(
+    client: Any,
+    prompt: str,
+    stream: Any,
+    *,
+    on_chunk: Any = None,
+    session: str | None = None,
+) -> dict[str, Any]:
+    """One turn, for every surface that runs one.
+
+    The CLI loop and the UI's `Session.ask` were the same loop written twice,
+    and only one of them checked the budget or recorded what the turn spent --
+    so everything done through the desktop app, which is the primary surface,
+    accrued no tokens in `ledger/quota.jsonl` and passed no ceiling. The README
+    said the allocation is checked "before issuing the next turn"; that was true
+    of `python agent.py` and false of `python agent.py --ui`. One driver, so
+    there is one answer.
+
+    `on_chunk` is called with each newly-visible piece of text; the UI passes
+    nothing because its renderer reads `stream.blocks` on a timer instead.
+    """
     refusal = check_turn_budget()
     if refusal:
-        print(f"\n[grad] {refusal['message']}\n[grad] fix: {refusal['fix']}", file=sys.stderr)
-        return False
+        raise BudgetRefused(refusal)
 
     await client.query(prompt)
-    stream = TurnStream()
-    async for message in client.receive_response():
-        # Whatever has not been printed yet -- a token as it arrives, the tail
-        # of a message that was never streamed, or a line naming a tool call.
-        # Never both halves of the same text.
-        chunk = stream.feed(message)
-        if chunk:
-            print(chunk, end="", flush=True)
-        usage = getattr(message, "usage", None)
-        if usage is not None:
-            quota_log.from_sdk_usage(
-                quota_log.STAGE_MAIN, usage, model=None, role="research"
+    sdk_session_id: str | None = None
+    last_usage: Any = None
+    recorded = None
+    try:
+        async for message in client.receive_response():
+            # Whatever has not been printed yet -- a token as it arrives, the
+            # tail of a message that was never streamed, or a line naming a tool
+            # call. Never both halves of the same text.
+            chunk = stream.feed(message)
+            if chunk and on_chunk is not None:
+                on_chunk(chunk)
+            # Captured from the stream rather than asked for: the SDK assigns
+            # it, and this is the id `resume` takes when a session is reopened.
+            # A resumed conversation can be given a new id, so the latest wins.
+            candidate = getattr(message, "session_id", None)
+            if isinstance(candidate, str) and candidate:
+                sdk_session_id = candidate
+            # The *last* usage seen, recorded once after the loop -- not one
+            # record per message. `ResultMessage` arrives last and carries the
+            # turn's cumulative usage, so summing every message that has a
+            # `usage` attribute would count the same tokens twice.
+            usage = getattr(message, "usage", None)
+            if usage is not None:
+                last_usage = usage
+    finally:
+        # In a `finally` because a turn that died half-way still spent what it
+        # spent. Letting the exception skip this would make a failing session
+        # the cheapest way to run untracked -- the accounting would be missing
+        # exactly the turns most worth accounting for.
+        if last_usage is not None:
+            recorded = quota_log.from_sdk_usage(
+                quota_log.STAGE_MAIN, last_usage, model=None, role="research", session=session
             )
+    return {"sdk_session_id": sdk_session_id, "quota": recorded}
+
+
+async def _turn(client: Any, prompt: str) -> bool:
+    """Run one turn. Returns False if the budget refused it."""
+    stream = TurnStream()
+    try:
+        await drive_turn(
+            client, prompt, stream, on_chunk=lambda c: print(c, end="", flush=True)
+        )
+    except BudgetRefused as exc:
+        print(
+            f"\n[grad] {exc.refusal['message']}\n[grad] fix: {exc.refusal['fix']}",
+            file=sys.stderr,
+        )
+        return False
     print()
     return True
 

core/budget.py

@@ -141,6 +141,17 @@ def projects() -> dict[str, dict[str, Any]]:
             continue
         kind = rec.get("type")
         if kind == T_PROJECT:
+            # First create wins. A duplicate -- two `budget new --id X` racing,
+            # or a stray line -- used to replace the fold wholesale, so the later
+            # record's ceilings won and the raise history vanished: an
+            # append-only ledger whose fold was last-writer-wins for the one
+            # record type that defines a ceiling. `create` refuses duplicates
+            # inside the append lock now; this is the backstop for the ones
+            # already written.
+            if pid in folded:
+                folded[pid].setdefault("duplicate_creates", 0)
+                folded[pid]["duplicate_creates"] += 1
+                continue
             folded[pid] = {
                 "id": pid,
                 "created_at": rec.get("created_at"),
@@ -194,6 +205,19 @@ def create(
             f"project {project_id!r} already exists",
             fix=f"python -m tools.budget status --project {project_id} --json",
         )
+
+    def _still_absent() -> None:
+        # Inside the append lock, like the expectation binding and the campaign
+        # halt. The check above runs first for the better message; this is what
+        # makes it atomic with the write, so two `budget new --id X` racing
+        # cannot both land -- the second record would otherwise redefine the
+        # first's ceilings.
+        if project_id in projects():
+            raise UsageError(
+                f"project {project_id!r} was created while this one was being written",
+                fix=f"python -m tools.budget status --project {project_id} --json",
+            )
+
     record = {
         "type": T_PROJECT,
         "id": project_id,
@@ -203,7 +227,7 @@ def create(
         "budget": {k: float(v) for k, v in budget.items() if v is not None},
         "status": "open",
     }
-    jsonl.append(projects_path(), record)
+    jsonl.append(projects_path(), record, precondition=_still_absent)
     return record
 
 
@@ -341,8 +365,19 @@ def status(project_id: str) -> dict[str, Any]:
             "ceiling": ceiling,
             "spent": consumed,
             "remaining": None if ceiling is None else round(float(ceiling) - consumed, 6),
+            # `ceiling is None`, not `not ceiling`: a project deliberately
+            # budgeted at zero ("no GPU spend on this one") has a real ceiling,
+            # and reporting `fraction: None` for it meant the Stop hook's
+            # threshold warnings skipped it entirely. A zero ceiling with
+            # anything spent is at 100%, not at "unbounded".
             "fraction": (
-                None if not ceiling else min(1.0, consumed / float(ceiling))
+                None
+                if ceiling is None
+                else (
+                    min(1.0, consumed / float(ceiling))
+                    if float(ceiling) > 0
+                    else (1.0 if consumed > 0 else 0.0)
+                )
             ),
             "over": bool(ceiling is not None and consumed > float(ceiling)),
         }

core/campaign.py

@@ -111,7 +111,40 @@ def escaped_evolve_block(baseline: str, candidate: str) -> dict[str, Any]:
     Whitespace-only differences outside the block do not count as an escape:
     a reformatter is not an environment change, and a check that fires
     spuriously is a check that gets argued around (§6).
+
+    **The markers are not evidence about themselves.** Each side's "outside" was
+    computed from its own markers, so a mutation that wrapped injected code --
+    new imports, a file write, an environment change -- in a *fresh*
+    `EVOLVE-BLOCK-START`/`END` pair moved that code into `inside` and left the
+    two outsides identical, and the escape check reported no escape. An LLM
+    mutation operator imitating the marker syntax it can see in its input is a
+    realistic accident, not just an attack. So the marker structure itself has
+    to match the baseline's before the outside comparison means anything.
     """
+    base_starts, base_ends = baseline.count(BLOCK_START), baseline.count(BLOCK_END)
+    cand_starts, cand_ends = candidate.count(BLOCK_START), candidate.count(BLOCK_END)
+
+    if cand_starts != cand_ends:
+        return {
+            "escaped": True,
+            "reason": (
+                f"the candidate has {cand_starts} EVOLVE-BLOCK-START marker(s) and "
+                f"{cand_ends} END marker(s); an unbalanced file has no well-defined "
+                "mutable region"
+            ),
+            "requires": "smoke",
+        }
+    if (cand_starts, cand_ends) != (base_starts, base_ends):
+        return {
+            "escaped": True,
+            "reason": (
+                f"the mutation changed the number of EVOLVE-BLOCK regions "
+                f"({base_starts} -> {cand_starts}); new markers can hide changed code "
+                "from this check, so the region structure is fixed by the baseline"
+            ),
+            "requires": "smoke",
+        }
+
     _, base_outside = split_blocks(baseline)
     _, cand_outside = split_blocks(candidate)