Implement HANDOFF-2: budgets, org namespaces, and the two loops

.gitignore

@@ -10,14 +10,38 @@ ledger/ledger.sqlite
 ledger/ledger.sqlite-*
 
 # Local corpus and artifacts. The JSONL ledgers ARE tracked on purpose:
-# they are the source of truth and are meant to be human-diffable.
+# they are the source of truth and are meant to be human-diffable --
+# projects.jsonl, campaigns.jsonl, and candidates.jsonl included.
 data/corpus.sqlite
 data/corpus.sqlite-*
 data/papers/
 ledger/runs/
 figures/
 notebooks/.ipynb_checkpoints/
 
+# Which project is selected is machine-local state, not a fact about the
+# research (HANDOFF-2 §15). Two checkouts should not fight over it.
+ledger/.current_project
+
+# Lab server state: a live port, pid, and token. Never committed.
+data/lab/
+
+# RepoWiki output. Generated, human-facing, and deliberately not competing with
+# the hand-written docs (§20).
+data/wiki/
+
+# LaTeX build byproducts. reports/<project>/main.tex, claims.json, and
+# references.bib ARE tracked: they are the checkable artifacts (§22).
+reports/**/*.pdf
+reports/**/*.aux
+reports/**/*.bbl
+reports/**/*.blg
+reports/**/*.fls
+reports/**/*.log
+reports/**/*.out
+reports/**/*.fdb_latexmk
+reports/**/claims.tex
+
 # Credentials never live in the workspace (HANDOFF §9). This is belt and braces.
 .env
 *.pem

README.md

@@ -3,9 +3,10 @@
 A personal research agent for mathematics and machine learning. Runs on one
 person's Windows desktop, backed by a Claude Max subscription.
 
-This is the implementation of [`HANDOFF.md`](HANDOFF.md), which remains the
-design document of record. Where this README and the handoff disagree, the
-handoff is the intent and this file is the report.
+This is the implementation of [`HANDOFF.md`](HANDOFF.md) and its extension
+[`HANDOFF-2.md`](HANDOFF-2.md), which together remain the design documents of
+record. Where this README and the handoffs disagree, the handoffs are the intent
+and this file is the report.
 
 The design temperament is borrowed from Mario Zechner's *pi*: trust the model,
 keep the system prompt small, keep the tool surface small, prefer files and CLIs
@@ -28,13 +29,36 @@ is a sentence in `prompts/system.md`.
 | Notebooks run clean top-to-bottom | `tools/nb.py verify` on a fresh kernel |
 | No general remote-execution capability | credentials in Windows Credential Manager, read only by `jobs.py` / `gpu.py` |
 | Concurrent ledger writes don't corrupt | one locked `core/jsonl.py:append`; no CLI writes a ledger file directly |
+| Token and credit spend stays bounded, not merely measured | `core/budget.py`, checked at every gateable event |
+| An evolutionary campaign cannot outspend its allocation | the campaign gate in `tools/evolve.py`, before generation 0 and before each generation after it |
+| A job submitted to an org is collectable from that org | the namespace is persisted on the run handle, not just passed at submit |
+| Every number in a report traces to a run record | `tools/report.py check` refuses on an unresolved claim |
+| Every citation in a report is a real paper | `report cite` resolves only against the corpus and verified S2 ids |
+| A result that has not been judged cannot be published | `report check` refuses while any cited run has an unjudged deviation |
+
+### The one thing that is *not* fully mechanical, and why
+
+**Subscription tokens are enforced to a granularity of one turn's overrun.**
+Tokens are consumed continuously inside a turn and there is no way to refuse
+mid-turn, so `agent.py` checks the remaining allocation *before* issuing the
+next turn and `hooks.py` denies cost-bearing Bash once the project is over. The
+turn that crosses the ceiling finishes.
+
+A second honesty note: subscription quota is not linear in tokens, and the real
+limits are rolling windows (5-hour and weekly on Max) that the SDK does not
+expose as a remaining balance. **A token ceiling is a proxy you control, not a
+mirror of Anthropic's limit.** The meter says so on screen.
 
 ## Install
 
 ```bash
 pip install -e ".[agent,notebook,retrieval,remote,ui,math,dev]"
 ```
 
+Optional extras, each pinned and each independently skippable: `lab` (the
+embedded JupyterLab, pinned exactly because the 3→4 break is what killed the
+Tabnine extension), `wiki` (RepoWiki), `evolve` (ShinkaEvolve).
+
 The core — ledger, preflight, gates, submitters — needs only the standard
 library plus a file lock. Everything heavier is optional and imported at the
 point of use, so `preflight` can refuse a submission on a machine with no
@@ -57,6 +81,7 @@ Store credentials once; they never enter the agent's environment:
 python -m tools.jobs credential set hf_token
 python -m tools.jobs credential set openrouter_key
 python -m tools.jobs credential set voyage_key
+python -m tools.jobs credential set context7_key   # optional; raises rate limits
 ```
 
 ## Run
@@ -87,7 +112,13 @@ that carry the literal next command.
 | `tools/jobs.py` | Hugging Face Jobs: `submit` / `status` / `collect` / `ceilings` / `credential` |
 | `tools/gpu.py` | the same verbs against a known SSH host |
 | `tools/ledger.py` | `expect` / `query` / `verdict` / `falsify` / `verify` / `reindex` |
-| `tools/quota.py` | measured token and credit usage, summarised by stage |
+| `tools/quota.py` | measured token and credit usage, summarised by stage, role, and project |
+| `tools/budget.py` | projects and their ceilings: `new` / `use` / `status` / `raise` / `close` |
+| `tools/docs.py` | is this library call current? introspection first, then Context7 |
+| `tools/evolve.py` | evolutionary search as a budgeted campaign, over ShinkaEvolve |
+| `tools/report.py` | `draft` / `write` / `cite` / `check` / `build` — the report and its gate |
+| `tools/lab.py` | the embedded JupyterLab server (human editing surface) |
+| `tools/wiki.py` | RepoWiki over `core/` and `tools/` — **human-facing only**, not an agent tool |
 
 ### Exit codes
 
@@ -108,10 +139,17 @@ things, and the model should not have to read prose to tell them apart.
 | 9 | a check ran and failed |
 | 10 | job still running (not an error) |
 | 11 | configuration or credential problem |
+| 12 | **gate**: project budget exceeded |
+
+12 is deliberately distinct from 6: "this research ran out of its allocation" is
+not "the machine is out of money", and conflating them makes the wrong fix look
+right.
 
 ## A full cycle
 
 ```bash
+python -m tools.budget new --id proj-scaling-w2 --title "width vs depth" \
+    --gpu-usd 50 --quota-tokens 5e6 --credits-usd 10 --payer hf:myorg --use --json
 python -m tools.preflight run --spec pipeline/spec.toml --only tests,dry_run --json
 python -m tools.jobs submit --spec pipeline/spec.toml --smoke --json
 python -m tools.ledger expect --task scaling-w2 --quantity val_loss@1e9_tokens \
@@ -122,11 +160,14 @@ python -m tools.jobs submit --spec pipeline/spec.toml --expect exp-... --json
 python -m tools.jobs collect run-... --json
 python -m tools.ledger verdict run-... --quantity val_loss@1e9_tokens \
     --verdict bug --note 'lr schedule off by one step' --json
+python -m tools.report draft --project proj-scaling-w2 --json   # free, no model
+python -m tools.report check --project proj-scaling-w2 --json   # the gate
 ```
 
-Skip any of the first three and the fourth refuses, with the command you skipped
-in its `fix` field. `skills/preflight/SKILL.md` documents the submission spec
-format and what each check catches.
+Skip any of the preflight/expectation steps and `submit` refuses, with the
+command you skipped in its `fix` field. Skip the verdict and `report check`
+refuses. `skills/preflight/SKILL.md` documents the submission spec format and
+what each check catches.
 
 ## Layout
 
@@ -138,30 +179,77 @@ core/                 the machinery the CLIs share, so no tool can forget a rule
   cli.py              the §8 CLI contract, implemented once
   jsonl.py            the single locked write path to the ledgers
   submission.py       the resolved submission and its hash
-  gates.py            the four submit gates and the smoke carve-out
+  gates.py            the submit gates and the smoke carve-out
+  budget.py           the project dimension and its three ceilings
   ledger_store.py     event-folded runs, rolling spend, staleness, derived index
   submit.py           shared submitter machinery: record, collect, deviations
+  campaign.py         campaigns, candidates, and the evolve-block escape check
+  report.py           the claim and citation guarantees `report check` enforces
   corpus.py           FTS5 + vectors + reciprocal rank fusion
   haiku.py            funnel stages 0 and 3, via forced SDK tools
-  http.py             Semantic Scholar, rerank, embeddings
+  http.py             Semantic Scholar, rerank, embeddings, Context7
 tools/                the CLIs
-ui/                   NiceGUI app and the four widgets
+ui/                   NiceGUI app and the widgets
+config/jupyter/       the Lab server config: framing headers and overrides
 skills/               loaded on demand, not into the default context
-ledger/               expectations.jsonl, runs.jsonl, quota.jsonl, preflight records
+ledger/               expectations.jsonl, runs.jsonl, quota.jsonl, projects.jsonl,
+                      campaigns.jsonl, candidates.jsonl, preflight records
+reports/<project>/    main.tex, claims.json, references.bib, the PDF
 evals/retrieval.jsonl the arbiter for any change to retrieval
 ```
 
 ## Status
 
-Implemented and tested: the ledger, the submission hash, all four gates, the
-smoke caps, the CLI contract, the hook, the persistent kernel, and notebook
-verification. `pytest` covers these — 85 tests, no network, no SDK required.
+Implemented and tested: the ledger, the submission hash, every gate, the smoke
+caps, the CLI contract, the hook, the persistent kernel, notebook verification,
+the project dimension and its three ceilings, HF organization namespaces,
+library-currency checking, the campaign loop and its budget gate, and the report
+generator with all four of its rules. `pytest` covers these — 295 tests, no
+network, no SDK required.
 
 Implemented but not exercised against a live service: the HF Jobs backend, the
-SSH backend, Semantic Scholar, the OpenRouter reranker, Voyage embeddings, and
-the two Haiku funnel stages. They are written against the documented interfaces
-and fail with actionable errors rather than tracebacks, but a real credential
-and a real run are what will find the mismatches.
+SSH backend, Semantic Scholar, the OpenRouter reranker, Voyage embeddings, the
+two Haiku funnel stages, Context7, ShinkaEvolve, and RepoWiki. They are written
+against the documented interfaces and fail with actionable errors rather than
+tracebacks, but a real credential and a real run are what will find the
+mismatches.
+
+**Two of [HANDOFF-2 §23](HANDOFF-2.md)'s open questions are now closed:**
+
+- **Context7's REST endpoints** (§23 item 2) are verified against the live API:
+  `/api/v2/libs/search` returns `{"results": […]}`, and `/api/v2/context` with
+  `type=json` returns `{"codeSnippets": […]}`. They stay in `[docs]` config
+  because a third-party API can move; the client reads both the v2 and v1
+  response keys so a config change is sufficient either way.
+- **ShinkaEvolve exposes no per-candidate callback, and no per-generation entry
+  point either** (§23 item 1). `ShinkaEvolveRunner` has `run` and `run_async`,
+  both of which own the whole loop — which is the control the campaign budget
+  gate needs in order to re-check between generations. **This is the evidence
+  §21 said a fork should wait for.** `evolve run` refuses with that explanation
+  rather than handing control away with the budget unchecked;
+  `python -m tools.evolve capabilities --json` reports what it found.
+
+**Still open:**
+
+- **Whether `headless/claude` works against a Max subscription specifically** is
+  reported in Shinka's release notes and untested here.
+- **Historical records are left as `"unassigned"`** rather than retrofitted with
+  a project. Cheap to change while the ledger is small.
+- **Phase 2 of the campaign loop (remote evaluation) is not enabled.**
+  `--remote` is refused: the gate is proven locally first, because doing the
+  ledger work and the spend work simultaneously against live GPU jobs is how you
+  learn about exit 7 the hard way.
+
+**One correction to HANDOFF-2 itself.** §20 records `repowiki map` as taking
+`--format html --open`. The 0.3.1 wheel's `map` takes exactly one `path`,
+`--format text|json`, and has neither flag — so `tools/wiki.py` invokes it once
+per scope directory asking for JSON and renders the HTML itself.
+
+**`tools/docs.py check` imports the modules it inspects,** and importing runs
+their top-level code. That is inherent to the introspection oracle — a checker
+that does not import can only guess at what is installed. Run it on your own
+pipeline, not on a repository you just downloaded; the module docstring and
+`--help` both say so.
 
 Two things worth knowing before trusting them:
 

agent.py

@@ -30,6 +30,7 @@
 
 import hooks
 from core import config as config_mod, credentials, paths, quota_log
+from core.errors import EXIT_PROJECT_BUDGET
 
 BUILTIN_TOOLS = ["Read", "Write", "Edit", "Bash", "Glob", "Grep"]
 
@@ -64,7 +65,7 @@ def build_options(cfg: Any, *, permission_mode: str | None = None) -> Any:
         "Stop": [sdk.HookMatcher(hooks=[hooks.stop])],
     }
     return sdk.ClaudeAgentOptions(
-        model=str(cfg.get("agent", "model", "claude-opus-4-5")),
+        model=cfg.model_for("research"),
         system_prompt=system_prompt(),
         allowed_tools=BUILTIN_TOOLS,
         disallowed_tools=DENIED_TOOLS,
@@ -81,11 +82,20 @@ def preflight_environment() -> dict[str, Any]:
     so a stray export silently bills the Developer Platform instead of the
     subscription. It is removed here rather than warned about.
     """
+    from core import budget  # noqa: PLC0415
+
     removed = credentials.scrub_environment()
+    cfg = config_mod.load()
+    project_id = budget.current_project()
     return {
         "removed_env": removed,
         "oauth_token_present": bool(os.environ.get("CLAUDE_CODE_OAUTH_TOKEN")),
         "workspace": str(paths.root()),
+        "models": cfg.models(),
+        # Read from ledger/.current_project, not from the environment -- the
+        # scrub above is exactly why the selection is a file (§15).
+        "project": project_id,
+        "project_status": budget.status(project_id) if budget.exists(project_id) else None,
         "note": (
             "auth should be subscription-backed; confirm with `claude /status`. "
             "--bare mode does not read CLAUDE_CODE_OAUTH_TOKEN, so this runs non-bare."
@@ -106,9 +116,9 @@ async def run_session(prompt: str | None, *, once: bool) -> int:
 
     async with sdk.ClaudeSDKClient(options=build_options(cfg)) as client:
         if prompt:
-            await _turn(client, prompt)
+            ran = await _turn(client, prompt)
             if once:
-                return 0
+                return 0 if ran else EXIT_PROJECT_BUDGET
         while True:
             try:
                 # In a worker thread: a bare input() blocks the event loop, and
@@ -126,16 +136,69 @@ async def run_session(prompt: str | None, *, once: bool) -> int:
             await _turn(client, line)
 
 
-async def _turn(client: Any, prompt: str) -> None:
+def check_turn_budget() -> dict[str, Any] | None:
+    """Refuse the *next* turn when the project is out of token allocation.
+
+    HANDOFF-2 §15, and the honesty is the point: tokens are consumed
+    continuously inside a turn and there is no way to refuse mid-turn, so
+    **token budgets are enforced to a granularity of one turn's overrun.** This
+    check is our code end to end -- it depends on no SDK behaviour -- and it runs
+    before `query`, not after.
+
+    Returns a refusal payload, or None to proceed.
+    """
+    try:
+        from core import budget  # noqa: PLC0415
+
+        project_id = budget.current_project()
+        if not project_id or not budget.exists(project_id):
+            return None
+        state = budget.status(project_id)
+    except Exception:  # noqa: BLE001 - accounting must never strand a session
+        return None
+
+    tokens = state["resources"]["quota_tokens"]
+    if not tokens["over"]:
+        return None
+    overrun = tokens["spent"] - float(tokens["ceiling"])
+    return {
+        "project": project_id,
+        "resource": "quota_tokens",
+        "spent": tokens["spent"],
+        "ceiling": tokens["ceiling"],
+        "overrun": overrun,
+        "message": (
+            f"project {project_id} has used {tokens['spent']:,} of its "
+            f"{int(tokens['ceiling']):,} token allocation -- {overrun:,.0f} over. "
+            "Refusing the next turn; the turn that crossed the ceiling was allowed to "
+            "finish, because there is no way to refuse mid-turn."
+        ),
+        "fix": (
+            f"python -m tools.budget raise --project {project_id} "
+            "--quota-tokens <new ceiling> --json"
+        ),
+    }
+
+
+async def _turn(client: Any, prompt: str) -> bool:
+    """Run one turn. Returns False if the budget refused it."""
+    refusal = check_turn_budget()
+    if refusal:
+        print(f"\n[grad] {refusal['message']}\n[grad] fix: {refusal['fix']}", file=sys.stderr)
+        return False
+
     await client.query(prompt)
     async for message in client.receive_response():
         text = _text_of(message)
         if text:
             print(text, end="", flush=True)
         usage = getattr(message, "usage", None)
         if usage is not None:
-            quota_log.from_sdk_usage(quota_log.STAGE_MAIN, usage)
+            quota_log.from_sdk_usage(
+                quota_log.STAGE_MAIN, usage, model=None, role="research"
+            )
     print()
+    return True
 
 
 def _text_of(message: Any) -> str: