From d16f47dc3b009021f4303959fe0c70cc14b4bf18 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 00:44:46 +0000 Subject: [PATCH 1/3] fix(runtime-witness): record the execution state, so "did not look" survives the process MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The exit-code contract keeps three states apart on purpose — 0 evaluated / witness absent, 1 evaluated / witness present, 2 not evaluated — because *not looking* and *looking and finding nothing* are different outcomes, and collapsing them is how a monitoring pipeline learns to report health it never measured. Persistence then collapsed exactly that distinction. A refused attach wrote no `runtime.json` at all, so after the process exited the durable record read: artifact present -> some evaluated outcome artifact absent -> not evaluated OR never invoked OR runner died before invocation OR persistence failed OR lost in transit OR an older format nothing reads any more Absence had too many preimages to carry meaning. Now it carries none: Absence of a record means no durable knowledge, never a semantic outcome. Every attempted evaluation writes a record when `--out` is given, stating what happened in `execution.state`: `observed` / `clean` (with the observation scope), `not_evaluated` (with a reason code), `error` (with a classification). What does NOT come back is a verdict nobody earned. A `not_evaluated` or `error` record carries no `verdict` and no `retained` key at all — not even `retained: []`, which downstream reads as "looked, found nothing" and would re-create the collapse one layer up. Two distinctions the record refuses to guess at: * `refused-attach` is a claim about permission, so it is made only where a refusing policy can be named (`reason.policy`, e.g. the Yama scope). Every other unreadable target gets the weaker, true `unreadable-target`. The human advice and the record now read one shared observation, so a diagnostic that blames the kernel can no longer sit beside a record that blames the target. * `not_evaluated` vs `error` splits on where the failure landed: before the heap was readable nothing was looked at and the target is not implicated; after it, the walk broke and the target is not exonerated. `scope` is required for an evaluated state, not decorative: a `clean` that does not say what it looked at cannot mean "nothing was there", and consumers must route it down the schema-violation path. It names the population the verdict covered separately from the budgets that bounded only the display, so a reader can tell a number that constrained the verdict from one that did not. CI moves with the contract. The denied-attach gate asserted "no artifact written" — the behaviour being fixed — and now asserts the refusal is recorded, names the policy, and carries no verdict. Two target-free assertions run on every platform, and the flagship demo checks that both its variants record a scope. Verified: witness selftest (classifier, verdict and record contracts), both flagship demo variants against a live net8 process (observed with a 1000-instance scope; clean), all three not-evaluated codes, ingest selftest 26/26, tests/run_tests.py wpf 28/28. Refs #331. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016Lmcv3X9PoELp8CGDfNc9m --- .github/workflows/ci.yml | 80 +++++- audit/runtime/RetentionPath/Program.cs | 364 ++++++++++++++++++++----- docs/runtime-witness-operations.md | 73 ++++- scripts/flagship-demo.sh | 17 ++ 4 files changed, 465 insertions(+), 69 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e6c89e40..3da84255 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -2715,11 +2715,57 @@ jobs: [ "$rc" -eq 2 ] || { echo "FAIL: bare usage must exit 2 (never clean), got $rc"; exit 1; } echo "$out" | grep -q "RETAINED (root path shown) | OBSERVED_ONLY" || { echo "FAIL: usage must document the verdict vocabulary"; exit 1; } set +e - out=$(dotnet "$GITHUB_WORKSPACE"/audit/runtime/RetentionPath/bin/Release/net8.0/RetentionPath.dll roots --pid 999999 --type X 2>&1) + out=$(dotnet "$GITHUB_WORKSPACE"/audit/runtime/RetentionPath/bin/Release/net8.0/RetentionPath.dll roots --pid 999999 --type X \ + --out "$RUNNER_TEMP/nopid.json" 2>&1) rc=$? set -e echo "$out" [ "$rc" -eq 2 ] || { echo "FAIL: a failed attach must exit 2, never read as clean, got $rc"; exit 1; } + # Every platform, no target needed: a run that could not look records + # that it could not look. Absence of a file is not that statement — + # it is also "never invoked" and "artifact lost", so it means nothing. + cat "$RUNNER_TEMP/nopid.json" + python3 - "$RUNNER_TEMP/nopid.json" unreadable-target <<'PY' + import json, sys + doc = json.load(open(sys.argv[1], encoding="utf-8")) + ex = doc.get("execution") or {} + problems = [] + if ex.get("state") != "not_evaluated": + problems.append(f"execution.state {ex.get('state')!r}, want 'not_evaluated'") + if (ex.get("reason") or {}).get("code") != sys.argv[2]: + problems.append(f"reason.code {(ex.get('reason') or {}).get('code')!r}, want {sys.argv[2]!r}") + if not (ex.get("reason") or {}).get("detail"): + problems.append("a not_evaluated record must carry a reason detail") + for key in ("verdict", "retained"): + if key in doc: + problems.append(f"a run that did not look must not record {key!r}") + for p in problems: + print(f"FAIL: {p}", file=sys.stderr) + sys.exit(1 if problems else 0) + PY + # A usage error is equally a state the record has to carry. + set +e + dotnet "$GITHUB_WORKSPACE"/audit/runtime/RetentionPath/bin/Release/net8.0/RetentionPath.dll roots \ + --out "$RUNNER_TEMP/usage.json" > /dev/null 2>&1 + rc=$? + set -e + [ "$rc" -eq 2 ] || { echo "FAIL: a usage error must exit 2, got $rc"; exit 1; } + python3 - "$RUNNER_TEMP/usage.json" usage-error <<'PY' + import json, sys + doc = json.load(open(sys.argv[1], encoding="utf-8")) + ex = doc.get("execution") or {} + problems = [] + if ex.get("state") != "not_evaluated": + problems.append(f"execution.state {ex.get('state')!r}, want 'not_evaluated'") + if (ex.get("reason") or {}).get("code") != sys.argv[2]: + problems.append(f"reason.code {(ex.get('reason') or {}).get('code')!r}, want {sys.argv[2]!r}") + for key in ("verdict", "retained"): + if key in doc: + problems.append(f"a run that did not look must not record {key!r}") + for p in problems: + print(f"FAIL: {p}", file=sys.stderr) + sys.exit(1 if problems else 0) + PY dotnet "$GITHUB_WORKSPACE"/audit/runtime/RetentionPath/bin/Release/net8.0/RetentionPath.dll selftest - name: "WPF flagship on Windows: the witness names the window the hub is holding (A2/A3)" if: runner.os == 'Windows' @@ -2860,9 +2906,35 @@ jobs: || { echo "FAIL: the diagnostic must name the policy that refused"; exit 1; } echo "$out" | grep -q "NOT a verdict" \ || { echo "FAIL: the diagnostic must say it did not look"; exit 1; } - [ ! -s "$RUNNER_TEMP/denied.json" ] \ - || { echo "FAIL: a refused attach must not write a verdict artifact"; exit 1; } - echo "OK: denied attach -> exit 2, policy named, no artifact written" + # The exit code says "I did not look" for as long as the process lives; + # the record has to say it afterwards. An absent file cannot: it also + # means never invoked, runner died, or artifact lost in transit. So the + # refusal is RECORDED — while the verdict it never earned is not. + [ -s "$RUNNER_TEMP/denied.json" ] \ + || { echo "FAIL: a refused attach must still record that it did not look"; exit 1; } + cat "$RUNNER_TEMP/denied.json" + python3 - "$RUNNER_TEMP/denied.json" <<'PY' + import json, sys + doc = json.load(open(sys.argv[1], encoding="utf-8")) + ex = doc.get("execution") or {} + problems = [] + if ex.get("state") != "not_evaluated": + problems.append(f"execution.state {ex.get('state')!r}, want 'not_evaluated'") + reason = ex.get("reason") or {} + if reason.get("code") != "refused-attach": + problems.append(f"reason.code {reason.get('code')!r}, want 'refused-attach'") + if "ptrace_scope" not in str(reason.get("policy", "")): + problems.append(f"reason.policy must name the refuser, got {reason.get('policy')!r}") + # The half that must NOT come back: an unearned verdict, or an empty + # `retained` that reads downstream as "looked, found nothing". + for key in ("verdict", "retained"): + if key in doc: + problems.append(f"a refused attach must not record {key!r} (got {doc[key]!r})") + for p in problems: + print(f"FAIL: {p}", file=sys.stderr) + sys.exit(1 if problems else 0) + PY + echo "OK: denied attach -> exit 2, policy named, refusal recorded, no verdict" - name: "flagship demo orchestrator end-to-end: bad DEMONSTRATED, ok VERIFIED (A3/A4)" if: runner.os == 'Linux' run: | diff --git a/audit/runtime/RetentionPath/Program.cs b/audit/runtime/RetentionPath/Program.cs index 146b9293..7dec3625 100644 --- a/audit/runtime/RetentionPath/Program.cs +++ b/audit/runtime/RetentionPath/Program.cs @@ -40,7 +40,7 @@ internal static class Program { private static int Main(string[] args) { - if (args.Length == 0) return Usage(); + if (args.Length == 0) return Usage(args, "no verb given"); string verb = args[0].ToLowerInvariant(); // The classifier boundary, pinned without a heap: the live net8 @@ -49,25 +49,34 @@ private static int Main(string[] args) if (verb == "selftest") return ClassifierSelfTest() ? 0 : 1; + if (verb != "census" && verb != "roots") + return Usage(args, $"unknown verb '{verb}'"); + int pid = ArgInt(args, "--pid", 0); string? dump = Arg(args, "--dump"); if (pid == 0 && dump == null) { Console.Error.WriteLine("retention-path: need --pid or --dump "); - return 2; + return NotEvaluated(args, "usage-error", "neither --pid nor --dump was given"); } + // Where the failure happened decides what it MEANS. Before the walker + // exists nothing has been read, so the failure is about the request or + // the target — `not_evaluated`. After it exists the heap was readable + // and the witness broke while looking, which is a different admission + // (`error`) and must not be dressed up as a polite refusal. + bool attached = false; try { using var walker = dump != null ? RetentionWalker.LoadDump(dump) : RetentionWalker.AttachToProcess(pid); + attached = true; switch (verb) { case "census": return Census(walker, args); - case "roots": return Roots(walker, args); - default: return Usage(); + default: return Roots(walker, args); } } catch (Exception ex) @@ -79,7 +88,23 @@ private static int Main(string[] args) { Console.Error.WriteLine(line); } - return 2; + if (attached) return Failed(args, ex); + + // `refused-attach` is a claim about PERMISSION, so it is made + // only where a refusing policy can be named. Everywhere else the + // honest statement is the weaker one — the target could not be + // read — with the exception carried in `detail`. Guessing + // "refused" from a process that merely still exists would assert + // something nobody observed, which is the failure mode this + // whole record exists to close. + string? policy = RefusingPolicy(pid, live: dump == null); + var reason = new Dictionary + { + ["code"] = policy != null ? "refused-attach" : "unreadable-target", + ["detail"] = $"{ex.GetType().Name}: {ex.Message}", + }; + if (policy != null) reason["policy"] = policy; + return NotEvaluated(args, reason); } } @@ -98,15 +123,8 @@ private static int Main(string[] args) /// private static IEnumerable AttachAdvice(int pid, bool live) { - if (!live || !OperatingSystem.IsLinux()) yield break; - - try { using var _ = System.Diagnostics.Process.GetProcessById(pid); } - catch { yield break; } // no such process: not a permission story - - string scope; - try { scope = File.ReadAllText("/proc/sys/kernel/yama/ptrace_scope").Trim(); } - catch { yield break; } // no Yama on this kernel - if (scope == "0") yield break; + string? scope = YamaScope(pid, live); + if (scope == null) yield break; yield return " the target is alive, so this is a PERMISSION failure: the kernel's"; yield return $" Yama policy (/proc/sys/kernel/yama/ptrace_scope = {scope}) refused it."; @@ -148,6 +166,34 @@ private static IEnumerable AttachAdvice(int pid, bool live) } } + /// The one place that decides whether a refusal was OBSERVED: + /// a live attach, on Linux, to a process that still exists, under a Yama + /// policy that is actually restricting. Returns the scope value, or null + /// when nothing here can be named as the refuser. Both the human advice + /// and the durable record read this — a diagnostic that blames the + /// kernel while the record blames the target would be two opinions about + /// one event. + private static string? YamaScope(int pid, bool live) + { + if (!live || !OperatingSystem.IsLinux()) return null; + + try { using var _ = System.Diagnostics.Process.GetProcessById(pid); } + catch { return null; } // no such process: not a permission story + + string scope; + try { scope = File.ReadAllText("/proc/sys/kernel/yama/ptrace_scope").Trim(); } + catch { return null; } // no Yama on this kernel + return scope == "0" ? null : scope; + } + + /// The refusing policy, named as the record must name it, or + /// null when no policy can be shown to have refused. + private static string? RefusingPolicy(int pid, bool live) + { + string? scope = YamaScope(pid, live); + return scope == null ? null : $"kernel.yama.ptrace_scope={scope}"; + } + private static int Census(RetentionWalker walker, string[] args) { var c = walker.Census(); @@ -166,36 +212,43 @@ private static int Census(RetentionWalker walker, string[] args) foreach (var kv in c.ByType.OrderByDescending(k => k.Value.Bytes).Take(top)) Console.WriteLine($"{Short(kv.Key),-62}{kv.Value.Count,14:N0}{Mb(kv.Value.Bytes),12:N1}"); - string? outPath = Arg(args, "--out"); - if (outPath != null) - { - // The runtime.json contract. `expected` is left at 0 — the collector does not - // know the budget; the scenario/config does, and correlate.py applies it. - var retained = c.ByType - .OrderByDescending(k => k.Value.Bytes) - .Take(top) - .Select(kv => new Dictionary - { - ["type"] = kv.Key, - ["count"] = kv.Value.Count, - ["expected"] = 0, - ["bytes"] = kv.Value.Bytes, - ["roots"] = new object[0], - }) - .ToList(); - - var doc = new Dictionary + // The runtime.json contract. `expected` is left at 0 — the collector does not + // know the budget; the scenario/config does, and correlate.py applies it. + var retained = c.ByType + .OrderByDescending(k => k.Value.Bytes) + .Take(top) + .Select(kv => new Dictionary { - ["schema"] = "own-runtime/1", - ["collector"] = CollectorIdentity(args), - ["retained"] = retained, - }; - File.WriteAllText(outPath, JsonConvert.SerializeObject(doc, Formatting.Indented)); - Console.WriteLine(); - Console.WriteLine($"runtime.json written to {outPath}"); - } + ["type"] = kv.Key, + ["count"] = kv.Value.Count, + ["expected"] = 0, + ["bytes"] = kv.Value.Bytes, + ["roots"] = new object[0], + }) + .ToList(); + + // Same exit-code tiers as `roots`, so the same execution states: + // a majority-retained heap is something OBSERVED and worth a `roots` + // run; anything less is a clean look, not a silent one. + bool present = c.RetainedShare > 50; + var scope = new Dictionary + { + ["verb"] = "census", + ["mode"] = Arg(args, "--dump") != null ? "dump" : "attach", + ["roots_enumerated"] = c.Roots, + ["objects_on_heap"] = c.HeapObjects, + ["objects_reachable"] = c.RetainedObjects, + ["bytes_on_heap"] = c.HeapBytes, + ["bytes_reachable"] = c.RetainedBytes, + ["retained_share_pct"] = Math.Round(c.RetainedShare, 1), + ["types_on_heap"] = c.ByType.Count, + ["types_reported"] = retained.Count, + ["top_budget"] = top, + }; + WriteRecord(args, BuildRecord( + CollectorIdentity(args), Evaluated(present, scope), retained: retained)); - return c.RetainedShare > 50 ? 1 : 0; + return present ? 1 : 0; } private static int Roots(RetentionWalker walker, string[] args) @@ -204,7 +257,7 @@ private static int Roots(RetentionWalker walker, string[] args) if (type == null) { Console.Error.WriteLine("retention-path roots: need --type "); - return 2; + return NotEvaluated(args, "usage-error", "roots requires --type "); } // Display budgets only — clamped to at least 1 so a pathological // `--sample 0` cannot suppress the root path a RETAINED verdict @@ -214,10 +267,11 @@ private static int Roots(RetentionWalker walker, string[] args) int maxHops = Math.Max(1, ArgInt(args, "--max-hops", 40)); var report = walker.FindRetainers(type, sample, maxHops); + var scope = RootsScope(args, type, report, sample, maxHops); if (report.TotalOnHeap == 0) { Console.WriteLine($"verdict: ABSENT — no instance of {type} is on the heap"); - WriteArtifact(args, "ABSENT", type, 0, new List()); + WriteArtifact(args, "ABSENT", type, 0, scope, new List()); return 0; } if (report.Retained == 0) @@ -227,7 +281,8 @@ private static int Roots(RetentionWalker walker, string[] args) Console.WriteLine($"verdict: OBSERVED_ONLY — {report.TotalOnHeap:N0} instance(s) of {type} on the " + "heap, but none of them is reachable from a GC root " + "(garbage awaiting collection, not an established retention)"); - WriteArtifact(args, "OBSERVED_ONLY", type, report.TotalOnHeap, new List()); + WriteArtifact(args, "OBSERVED_ONLY", type, report.TotalOnHeap, scope, + new List()); return 0; } @@ -253,7 +308,8 @@ private static int Roots(RetentionWalker walker, string[] args) Console.WriteLine($" via [{r.ContractKind()}], {r.Path.Count} hops:"); Console.Write(r.Render()); } - WriteArtifact(args, "OBSERVED_ONLY", type, report.TotalOnHeap, report.Retainers); + WriteArtifact(args, "OBSERVED_ONLY", type, report.TotalOnHeap, scope, + report.Retainers); return 0; } @@ -292,7 +348,8 @@ private static int Roots(RetentionWalker walker, string[] args) "really is held from many places"); } - WriteArtifact(args, "RETAINED", report.TypeName, report.TotalOnHeap, report.Retainers); + WriteArtifact(args, "RETAINED", report.TypeName, report.TotalOnHeap, scope, + report.Retainers); return 1; // retention found } @@ -309,20 +366,117 @@ internal static bool IsDurableKind(string kind) => internal static string VerdictOf(IEnumerable retainerKinds) => retainerKinds.Any(IsDurableKind) ? "RETAINED" : "OBSERVED_ONLY"; - /// The one `runtime.json` writer — every verdict emits the + /// The one `runtime.json` writer — every outcome emits the /// artifact when `--out` is given, so the ok-side of a demo is as - /// machine-checkable as the leak side. - private static void WriteArtifact( - string[] args, string verdict, string typeName, long count, IReadOnlyList retainers) + /// machine-checkable as the leak side, and a run that never looked is as + /// machine-checkable as one that did. + /// + /// The exit codes already keep three states apart (0 evaluated/absent, + /// 1 evaluated/present, 2 not evaluated) and then the process ends. If + /// storage represents "not evaluated" by writing nothing, that guarantee + /// does not survive it: an absent file also means never invoked, runner + /// died, persistence failed, artifact lost in transit, or a format + /// nothing reads any more. Absence has too many preimages to carry + /// meaning, so it is given none — the record IS the state. + /// + /// What must NOT come back is a verdict that was not earned: a + /// `not_evaluated` or `error` record carries no `verdict` and no + /// `retained` key at all. An empty `retained: []` would read downstream + /// as "looked, found nothing", which is the very collapse this record + /// exists to prevent. + private static void WriteRecord(string[] args, Dictionary doc) { string? outPath = Arg(args, "--out"); if (outPath == null) return; + File.WriteAllText(outPath, JsonConvert.SerializeObject(doc, Formatting.Indented)); + Console.WriteLine(); + Console.WriteLine($"runtime.json written to {outPath}"); + } + + /// Assemble the document. Pure, so the selftest can assert the + /// record contract without a heap, a process, or a filesystem. + internal static Dictionary BuildRecord( + Dictionary collector, + Dictionary execution, + string? verdict = null, + object? retained = null) + { var doc = new Dictionary { ["schema"] = "own-runtime/1", - ["verdict"] = verdict, - ["collector"] = CollectorIdentity(args), - ["retained"] = new object[] + ["execution"] = execution, + ["collector"] = collector, + }; + // Only an evaluated state may carry a measurement. + if (verdict != null) doc["verdict"] = verdict; + if (retained != null) doc["retained"] = retained; + return doc; + } + + /// The execution state of an evaluation that HAPPENED. `scope` + /// is required, not decorative: a `clean` whose scope is unknown is a + /// malformed record — it does not say what was looked at, so it cannot + /// mean "nothing was there" — and consumers must treat it as a schema + /// violation rather than as a quieter `not_evaluated`. + internal static Dictionary Evaluated( + bool witnessPresent, Dictionary scope) => + new Dictionary + { + ["state"] = witnessPresent ? "observed" : "clean", + ["scope"] = scope, + }; + + /// Record "I did not look, and here is why", then exit 2. + private static int NotEvaluated(string[] args, string code, string detail) => + NotEvaluated(args, new Dictionary + { + ["code"] = code, + ["detail"] = detail, + }); + + private static int NotEvaluated(string[] args, Dictionary reason) + { + WriteRecord(args, BuildRecord( + CollectorIdentity(args), + new Dictionary + { + ["state"] = "not_evaluated", + ["reason"] = reason, + })); + return 2; + } + + /// Record "I looked and broke", then exit 2. Distinct from + /// `not_evaluated` on purpose: the heap was readable, so a partial walk + /// may have happened and the target is not exonerated by this outcome. + /// The classification is the exception type — the honest granularity a + /// collector has, rather than a guess at a cause. + private static int Failed(string[] args, Exception ex) + { + WriteRecord(args, BuildRecord( + CollectorIdentity(args), + new Dictionary + { + ["state"] = "error", + ["error"] = new Dictionary + { + ["classification"] = ex.GetType().Name, + ["detail"] = ex.Message, + ["phase"] = "walk", + }, + })); + return 2; + } + + private static void WriteArtifact( + string[] args, string verdict, string typeName, long count, + Dictionary scope, IReadOnlyList retainers) + { + var doc = BuildRecord( + CollectorIdentity(args), + Evaluated(witnessPresent: verdict == "RETAINED", scope: scope), + verdict, + new object[] { new Dictionary { @@ -340,13 +494,29 @@ private static void WriteArtifact( ["path"] = r.Path.Select(h => h.ToString()).ToList(), }).ToList(), }, - }, - }; - File.WriteAllText(outPath, JsonConvert.SerializeObject(doc, Formatting.Indented)); - Console.WriteLine(); - Console.WriteLine($"runtime.json written to {outPath}"); + }); + WriteRecord(args, doc); } + /// What the `roots` walk actually covered. Population figures + /// come from the exact whole-population census, and the budgets that + /// bounded only the DISPLAY are named as budgets — a reader must be able + /// to tell a number that constrained the verdict from one that did not. + private static Dictionary RootsScope( + string[] args, string typeName, RetentionReport report, int sample, int maxHops) => + new Dictionary + { + ["verb"] = "roots", + ["mode"] = Arg(args, "--dump") != null ? "dump" : "attach", + ["type"] = typeName, + ["instances_on_heap"] = report.TotalOnHeap, + ["instances_reachable"] = report.Retained, + ["instances_durably_retained"] = report.DurableRetained, + ["paths_resolved"] = report.PathsResolved, + ["sample_budget"] = sample, + ["max_hops_budget"] = maxHops, + }; + /// Who read the heap and how — so the artifact is auditable /// (A3): the target (pid or dump path), the collector runtime, the OS. /// No timestamps: identical heaps must yield identical artifacts. @@ -462,14 +632,80 @@ void Check(string name, string got, string want) $"IsTransientRootKind={transient}, IsDurableKind(Classify)={durable}"); } + // 7. The record contract (issue #331). The exit codes keep three + // states apart and then the process ends; these checks pin that + // the storage layer keeps them apart too, instead of letting + // file-absence stand in for "not evaluated" — an absence that + // also means never invoked, runner died, or artifact lost. + var collector = new Dictionary { ["tool"] = "retention-path" }; + var someScope = new Dictionary { ["verb"] = "roots" }; + + void CheckRecord(string name, Dictionary doc, + string wantState, bool wantMeasurement) + { + if (!doc.TryGetValue("execution", out var exObj) || + exObj is not Dictionary ex) + { + fails.Add($"{name}: record has no `execution` block"); + return; + } + Check($"{name}: state", ex.TryGetValue("state", out var s) ? $"{s}" : "", wantState); + + // Each state owes its own evidence. A state with nothing behind + // it is a label, and a label is what this record replaces. + string owes = wantState switch + { + "observed" or "clean" => "scope", + "not_evaluated" => "reason", + _ => "error", + }; + if (!ex.ContainsKey(owes)) + fails.Add($"{name}: state '{wantState}' must carry `{owes}`"); + + // The half that must NOT come back: an unearned verdict, or an + // empty `retained` that reads downstream as "looked, found + // nothing". Absence of the key is the point. + bool hasMeasurement = doc.ContainsKey("verdict") || doc.ContainsKey("retained"); + if (hasMeasurement != wantMeasurement) + fails.Add($"{name}: measurement keys present={hasMeasurement}, want {wantMeasurement}"); + } + + CheckRecord("observed record", + BuildRecord(collector, Evaluated(true, someScope), "RETAINED", new object[0]), + "observed", wantMeasurement: true); + CheckRecord("clean record", + BuildRecord(collector, Evaluated(false, someScope), "ABSENT", new object[0]), + "clean", wantMeasurement: true); + CheckRecord("not_evaluated record", + BuildRecord(collector, new Dictionary + { + ["state"] = "not_evaluated", + ["reason"] = new Dictionary { ["code"] = "refused-attach" }, + }), + "not_evaluated", wantMeasurement: false); + CheckRecord("error record", + BuildRecord(collector, new Dictionary + { + ["state"] = "error", + ["error"] = new Dictionary { ["classification"] = "IOException" }, + }), + "error", wantMeasurement: false); + + // The state names follow the exit-code tiers, so a consumer can map + // one onto the other without a second opinion about what happened. + Check("witness present is `observed`", + $"{Evaluated(true, someScope)["state"]}", "observed"); + Check("witness absent but evaluated is `clean`", + $"{Evaluated(false, someScope)["state"]}", "clean"); + foreach (var f in fails) Console.Error.WriteLine($"FAIL: classifier {f}"); if (fails.Count == 0) - Console.WriteLine("retention-path classifier selftest OK: 16 checks passed"); + Console.WriteLine("retention-path selftest OK: classifier, verdict and record contracts hold"); return fails.Count == 0; } - private static int Usage() + private static int Usage(string[] args, string detail) { Console.Error.WriteLine("usage:"); Console.Error.WriteLine(" RetentionPath selftest # classifier fixtures, no target needed"); @@ -479,7 +715,11 @@ private static int Usage() Console.Error.WriteLine(" census is there anything retained at all, or is the heap just uncollected garbage?"); Console.Error.WriteLine(" roots what holds the TYPICAL instance of a type (exact verdict; sampled, ranked paths);"); Console.Error.WriteLine(" verdicts: RETAINED (root path shown) | OBSERVED_ONLY (no path established) | ABSENT"); - return 2; + Console.Error.WriteLine(); + Console.Error.WriteLine(" --out write the runtime.json record. EVERY outcome writes one, including"); + Console.Error.WriteLine(" a run that never looked: execution.state is observed | clean |"); + Console.Error.WriteLine(" not_evaluated | error, and only an evaluated state carries a verdict."); + return NotEvaluated(args, "usage-error", detail); } } } diff --git a/docs/runtime-witness-operations.md b/docs/runtime-witness-operations.md index 74368656..efc929f3 100644 --- a/docs/runtime-witness-operations.md +++ b/docs/runtime-witness-operations.md @@ -32,11 +32,78 @@ forbidden, this is the answer, not a workaround. Exit 2 is the one that matters here. *Not looking* and *looking and finding nothing* are different outcomes, and collapsing them is how a monitoring -pipeline learns to report health it never measured. A refused attach also -writes **no** `runtime.json` artifact — there is no verdict to record. +pipeline learns to report health it never measured. **Proven by CI:** a denied attach exits 2, names the policy that refused, and -leaves no artifact behind. +records that it did not look — without recording a verdict. + +## The durable record + +An exit code lives as long as the process. Everything downstream reads the +`runtime.json` that `--out` writes, so the three states above have to survive +into storage or the guarantee ends with the process. + +Representing *"not evaluated"* by writing **no file** does not survive it. +An absent artifact means: + +```text +not evaluated OR never invoked OR the runner died before invocation + OR persistence failed OR lost in transit + OR an older format nothing reads any more +``` + +Absence has too many preimages to carry meaning, so it is given none: + +> **Absence of a record means no durable knowledge, never a semantic outcome.** + +Every attempted evaluation writes a record when `--out` is given, and the record +states what happened: + +| `execution.state` | Exit | Carries | Meaning | +| --- | --- | --- | --- | +| `observed` | 1 | `scope`, `verdict`, `retained` | The heap was read; a witness is present. | +| `clean` | 0 | `scope`, `verdict`, `retained` | The heap was read; no witness. | +| `not_evaluated` | 2 | `reason.code`, `reason.detail` | Nothing was read. No verdict is recorded. | +| `error` | 2 | `error.classification` | The heap was readable and the walk broke. | + +```jsonc +{ + "schema": "own-runtime/1", + "execution": { + "state": "not_evaluated", + "reason": { + "code": "refused-attach", // usage-error | unreadable-target | refused-attach + "detail": "ClrDiagnosticsException: Could not attach to process 4213", + "policy": "kernel.yama.ptrace_scope=1" // only when a refuser can be named + } + }, + "collector": { "tool": "retention-path", "mode": "attach", "target": "4213", … } +} +``` + +Two things this record deliberately does **not** do. + +It does not record a verdict it did not earn. A `not_evaluated` or `error` +record carries no `verdict` key and no `retained` key **at all** — not even +`retained: []`, which downstream reads as *"looked, found nothing"* and would +re-create the collapse one layer up. + +It does not claim a refusal it did not observe. `refused-attach` is a statement +about permission, so it is used only where a refusing policy can be named +(`reason.policy`); every other unreadable target gets the weaker, true +`unreadable-target` with the exception in `detail`. + +`not_evaluated` and `error` are separated by *where* the failure landed: before +the heap was readable, nothing was looked at and the target is not implicated; +after it, the witness itself broke mid-walk and the target is not exonerated. + +**A `clean` with no `scope` is malformed, not weak.** A record that does not say +what was looked at cannot mean "nothing was there", so consumers must route it +down the schema-violation path — never read it as a quieter `not_evaluated`. +`scope` names the population the verdict covered (`instances_on_heap`, +`instances_reachable`, `instances_durably_retained`) separately from the budgets +that bounded only the display (`sample_budget`, `max_hops_budget`), so a reader +can tell a number that constrained the verdict from one that did not. ## Linux: Yama's `ptrace_scope` diff --git a/scripts/flagship-demo.sh b/scripts/flagship-demo.sh index 0d5e35da..b6418b51 100755 --- a/scripts/flagship-demo.sh +++ b/scripts/flagship-demo.sh @@ -72,6 +72,21 @@ m = re.search(r"still subscribed", app) subs = re.search(r"(\d[\d,]*) still subscribed", app) problems = [] +def check_execution(doc, want_state): + """An evaluated record must say WHAT it looked at. A verdict whose scope is + unknown cannot mean 'nothing was there' — it is malformed, not merely + quiet, and gets the schema-violation path rather than a lenient read.""" + ex = doc.get("execution") or {} + if ex.get("state") != want_state: + problems.append(f"execution.state {ex.get('state')!r}, want {want_state!r}") + scope = ex.get("scope") + if not scope: + problems.append(f"an evaluated record without `scope` is malformed (state {ex.get('state')!r})") + return + for key in ("verb", "mode", "type", "instances_on_heap"): + if key not in scope: + problems.append(f"scope lacks {key!r}") + if variant == "bad": if rc != 1: problems.append(f"witness exit {rc}, want 1 (RETAINED)") @@ -82,6 +97,7 @@ if variant == "bad": doc = {} if doc.get("verdict") != "RETAINED": problems.append(f"JSON verdict {doc.get('verdict')!r}, want RETAINED") + check_execution(doc, "observed") if "verdict: RETAINED" not in human: problems.append("human output lacks 'verdict: RETAINED'") roots = (doc.get("retained") or [{}])[0].get("roots") or [] @@ -110,6 +126,7 @@ else: doc = {} if doc.get("verdict") not in ("ABSENT", "OBSERVED_ONLY"): problems.append(f"JSON verdict {doc.get('verdict')!r}, want ABSENT or OBSERVED_ONLY") + check_execution(doc, "clean") roots = (doc.get("retained") or [{}])[0].get("roots") or [] durable = [r.get("kind") for r in roots if r.get("kind") not in ("stack", "finalizer")] if durable: From f13b645cf0bf17cb6188cd0250eb06009de86da6 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 03:43:28 +0000 Subject: [PATCH 2/3] fix(runtime-witness): attribute a failure to the stage it happened at MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review of d16f47d found the refused-attach fix stopped one level short of its own proof, and in two places. **stderr could still blame Yama after a successful attach.** The catch block called AttachAdvice unconditionally and only then checked whether the walker existed, so any exception from Census/Roots on Linux under a restricting ptrace_scope printed "PERMISSION failure, Yama refused it" while the record correctly said `execution.state=error`. The claimed invariant — human diagnostic and record read one observation — was still broken, just in the other branch. **One bool could not carry the state it was asked to.** `attached` was set after RetentionWalker's constructor, which does DataTarget.AttachToProcess AND ClrVersions.FirstOrDefault AND CreateRuntime. A live process that opened fine and turned out not to be managed threw with attached still false, so under ptrace_scope=1/2/3 it came back labelled `refused-attach` — a permission claim about a step where permission had already been granted. The stages are now explicit — open-target, create-runtime, walk — and RetentionWalker splits accordingly (OpenLiveTarget/OpenDumpTarget, then Create taking ownership). Attribution follows the stage: only open-target may yield `refused-attach` and only it prints the ptrace advice; create-runtime is `unreadable-target`; walk is `error`. `reason.stage` is recorded so a reader can check the attribution rather than trust it. And the last substitution the previous round left standing: Yama being active is not Yama having caused this failure. The kernel does not tell a tracer which check rejected it, and a live process under scope 1 can fail to open for unrelated reasons. The field is now `policy_in_force` — what was observed — and the stderr wording says the same, naming a likely cause and its limit rather than a refuser. Also correcting two overclaims from d16f47d rather than leaving them to be found again. `verdict` is command-specific (roots has a vocabulary, census does not), so the doc no longer lists it among what every evaluated state carries — the code never wrote one for census. And the record invariant is stated as it is implemented: every evaluation *for which persistence was requested* leaves a record; a run without --out asked for no durable output, and a standalone tool has no other publication point. CI: the denied-attach gate now also asserts reason.stage, and a second case runs under the SAME restricting policy — an unreadable dump — proving a non-attach failure does not borrow the policy or the lecture. Verified: selftest (now pinning stage attribution as a pure function, including create-runtime never citing a policy), both flagship variants on a live net8 process, live unmanaged process -> create-runtime/unreadable-target with no PERMISSION line on stderr, dead pid -> open-target/unreadable-target, ingest 26/26, run_tests.py wpf 28/28. The live Yama refusal itself is unexercised here (no Yama in this kernel) and remains CI's to prove. Refs #331. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016Lmcv3X9PoELp8CGDfNc9m --- .github/workflows/ci.yml | 39 +++++- audit/runtime/RetentionPath/Heap.cs | 21 +++- audit/runtime/RetentionPath/Program.cs | 160 ++++++++++++++++++------- docs/runtime-witness-operations.md | 54 ++++++--- 4 files changed, 209 insertions(+), 65 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3da84255..a8c5251e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -2923,8 +2923,13 @@ jobs: reason = ex.get("reason") or {} if reason.get("code") != "refused-attach": problems.append(f"reason.code {reason.get('code')!r}, want 'refused-attach'") - if "ptrace_scope" not in str(reason.get("policy", "")): - problems.append(f"reason.policy must name the refuser, got {reason.get('policy')!r}") + # A permission claim belongs to the one stage a permission check applies + # to. Anything later opened the target fine and must not cite a policy. + if reason.get("stage") != "open-target": + problems.append(f"reason.stage {reason.get('stage')!r}, want 'open-target'") + if "ptrace_scope" not in str(reason.get("policy_in_force", "")): + problems.append("reason.policy_in_force must name the policy that was in force, " + f"got {reason.get('policy_in_force')!r}") # The half that must NOT come back: an unearned verdict, or an empty # `retained` that reads downstream as "looked, found nothing". for key in ("verdict", "retained"): @@ -2934,7 +2939,35 @@ jobs: print(f"FAIL: {p}", file=sys.stderr) sys.exit(1 if problems else 0) PY - echo "OK: denied attach -> exit 2, policy named, refusal recorded, no verdict" + # The other half, with the SAME restricting policy still in force: a + # failure that is not an attach must not borrow it. Yama being on is + # observable; Yama having caused the failure in hand is not, and the + # record may only say the first. + echo "not a dump" > "$RUNNER_TEMP/not-a-dump" + set +e + dotnet "$WITNESS" roots --dump "$RUNNER_TEMP/not-a-dump" --type X \ + --out "$RUNNER_TEMP/dumpfail.json" > "$RUNNER_TEMP/dumpfail.log" 2>&1 + drc=$? + set -e + cat "$RUNNER_TEMP/dumpfail.log" + [ "$drc" -eq 2 ] || { echo "FAIL: an unreadable dump must exit 2, got $drc"; exit 1; } + grep -q "ptrace_scope" "$RUNNER_TEMP/dumpfail.log" \ + && { echo "FAIL: a dump read must not lecture about ptrace"; exit 1; } + python3 - "$RUNNER_TEMP/dumpfail.json" <<'PY' + import json, sys + reason = ((json.load(open(sys.argv[1], encoding="utf-8")).get("execution") or {}) + .get("reason") or {}) + problems = [] + if reason.get("code") != "unreadable-target": + problems.append(f"reason.code {reason.get('code')!r}, want 'unreadable-target'") + if "policy_in_force" in reason: + problems.append(f"a dump read cited a ptrace policy: {reason['policy_in_force']!r}") + for p in problems: + print(f"FAIL: {p}", file=sys.stderr) + sys.exit(1 if problems else 0) + PY + echo "OK: denied attach -> exit 2, policy named, refusal recorded, no verdict;" + echo "OK: a non-attach failure under the same policy does not borrow it" - name: "flagship demo orchestrator end-to-end: bad DEMONSTRATED, ok VERIFIED (A3/A4)" if: runner.os == 'Linux' run: | diff --git a/audit/runtime/RetentionPath/Heap.cs b/audit/runtime/RetentionPath/Heap.cs index 67f71c12..1a1df95f 100644 --- a/audit/runtime/RetentionPath/Heap.cs +++ b/audit/runtime/RetentionPath/Heap.cs @@ -38,13 +38,24 @@ internal sealed class RetentionWalker : IDisposable private readonly DataTarget _target; private readonly ClrRuntime _runtime; - /// Attach to a LIVE process (suspends it for the read). No procdump needed. - public static RetentionWalker AttachToProcess(int pid) => - new RetentionWalker(DataTarget.AttachToProcess(pid, suspend: true)); + /// Attach to a LIVE process (suspends it for the read). No procdump needed. + /// + /// Opening the target is deliberately its OWN step, separate from + /// : it is the only one a kernel ptrace policy can + /// refuse. Folding the two together makes every CLR-initialisation + /// failure indistinguishable from a permission failure, and a caller + /// that cannot tell them apart will attribute one to the other. + public static DataTarget OpenLiveTarget(int pid) => + DataTarget.AttachToProcess(pid, suspend: true); /// Read a full dump — the right choice when the target must not be paused. - public static RetentionWalker LoadDump(string path) => - new RetentionWalker(DataTarget.LoadDump(path)); + public static DataTarget OpenDumpTarget(string path) => + DataTarget.LoadDump(path); + + /// Build the CLR view over an already-opened target. Takes + /// ownership: on success the walker disposes the target, and on failure + /// the target is still the caller's to dispose. + public static RetentionWalker Create(DataTarget target) => new RetentionWalker(target); private RetentionWalker(DataTarget target) { diff --git a/audit/runtime/RetentionPath/Program.cs b/audit/runtime/RetentionPath/Program.cs index 7dec3625..3d2dfbed 100644 --- a/audit/runtime/RetentionPath/Program.cs +++ b/audit/runtime/RetentionPath/Program.cs @@ -60,18 +60,27 @@ private static int Main(string[] args) return NotEvaluated(args, "usage-error", "neither --pid nor --dump was given"); } - // Where the failure happened decides what it MEANS. Before the walker - // exists nothing has been read, so the failure is about the request or - // the target — `not_evaluated`. After it exists the heap was readable - // and the witness broke while looking, which is a different admission - // (`error`) and must not be dressed up as a polite refusal. - bool attached = false; + // WHERE the failure happened decides what it means, so the stages are + // tracked separately instead of being collapsed into one "did we get + // in" flag. Opening the target is the only step a ptrace policy can + // refuse; building the CLR view happens after the target is already + // open, so its failures say nothing about permission; and a failure + // during the walk is the witness breaking, not the target refusing. + // One bool cannot carry that, and when it tried, a CLR-initialisation + // failure on a live process under a restricting Yama policy came back + // labelled `refused-attach`. + DataTarget? target = null; + var stage = Stage.OpenTarget; try { - using var walker = dump != null - ? RetentionWalker.LoadDump(dump) - : RetentionWalker.AttachToProcess(pid); - attached = true; + target = dump != null + ? RetentionWalker.OpenDumpTarget(dump) + : RetentionWalker.OpenLiveTarget(pid); + + stage = Stage.CreateRuntime; + using var walker = RetentionWalker.Create(target); + target = null; // the walker owns it from here + stage = Stage.Walk; switch (verb) { @@ -84,30 +93,71 @@ private static int Main(string[] args) // A failed read must not read as "clean" — exit 2, distinct from // 0 (analysed, nothing retained) and 1 (analysed, retention found). Console.Error.WriteLine($"retention-path: {ex.GetType().Name}: {ex.Message}"); - foreach (var line in AttachAdvice(pid, live: dump == null)) + + // Only the stage a policy could have refused gets the ptrace + // lecture. Printing it after the target opened would have stderr + // blaming the kernel while the record blames the walk — the two + // reading one observation is the whole point of sharing YamaScope. + if (stage == Stage.OpenTarget) { - Console.Error.WriteLine(line); + foreach (var line in AttachAdvice(pid, live: dump == null)) + Console.Error.WriteLine(line); } - if (attached) return Failed(args, ex); - - // `refused-attach` is a claim about PERMISSION, so it is made - // only where a refusing policy can be named. Everywhere else the - // honest statement is the weaker one — the target could not be - // read — with the exception carried in `detail`. Guessing - // "refused" from a process that merely still exists would assert - // something nobody observed, which is the failure mode this - // whole record exists to close. - string? policy = RefusingPolicy(pid, live: dump == null); - var reason = new Dictionary - { - ["code"] = policy != null ? "refused-attach" : "unreadable-target", - ["detail"] = $"{ex.GetType().Name}: {ex.Message}", - }; - if (policy != null) reason["policy"] = policy; - return NotEvaluated(args, reason); + + return stage == Stage.Walk + ? Failed(args, ex) + : NotEvaluated(args, ReadFailure(stage, pid, live: dump == null, ex)); + } + finally + { + // Non-null only when ownership never reached the walker. + target?.Dispose(); } } + /// The stage a run reached, because the same exception means + /// different things at each one. + private enum Stage + { + /// Opening the process or dump — refusable by a ptrace policy. + OpenTarget, + /// Building the CLR view over an already-open target. + CreateRuntime, + /// Reading the heap. + Walk, + } + + /// + /// Why the heap could not be read, said no more strongly than the + /// evidence allows. + /// + /// `refused-attach` is a claim about PERMISSION, so it is reserved for a + /// failure at the one stage a permission check applies to, with a + /// restricting policy actually in force. Even then the policy is recorded + /// as policy_in_force, not as the proven cause: a live process + /// under `ptrace_scope=1` can fail to open for reasons that have nothing + /// to do with Yama, and this collector cannot tell those apart. Naming + /// what was in force is observation; naming it as the refuser would be + /// the same unearned confidence the execution record exists to prevent. + /// + /// Everything else — including a target that opened and then turned out + /// not to be a readable CLR process — gets the weaker, true + /// `unreadable-target`, with the exception in `detail`. + /// + private static Dictionary ReadFailure( + Stage stage, int pid, bool live, Exception ex) + { + string? policy = stage == Stage.OpenTarget ? RefusingPolicy(pid, live) : null; + var reason = new Dictionary + { + ["code"] = policy != null ? "refused-attach" : "unreadable-target", + ["stage"] = stage == Stage.OpenTarget ? "open-target" : "create-runtime", + ["detail"] = $"{ex.GetType().Name}: {ex.Message}", + }; + if (policy != null) reason["policy_in_force"] = policy; + return reason; + } + /// /// Turn a bare ClrMD exception into something a person can act on when /// the kernel — not the tool — refused the attach. On Linux, Yama's @@ -126,8 +176,10 @@ private static IEnumerable AttachAdvice(int pid, bool live) string? scope = YamaScope(pid, live); if (scope == null) yield break; - yield return " the target is alive, so this is a PERMISSION failure: the kernel's"; - yield return $" Yama policy (/proc/sys/kernel/yama/ptrace_scope = {scope}) refused it."; + yield return " the target is alive and the open was refused, so a PERMISSION failure"; + yield return $" is the likely cause: the kernel's Yama policy is restricting"; + yield return $" (/proc/sys/kernel/yama/ptrace_scope = {scope}). That policy being in"; + yield return " force is what Owen can see; it cannot prove this open is what it stopped."; yield return " Owen did not look — this is NOT a verdict about the target's heap."; // Each mode restricts something different, and the remedies do not @@ -166,13 +218,16 @@ private static IEnumerable AttachAdvice(int pid, bool live) } } - /// The one place that decides whether a refusal was OBSERVED: - /// a live attach, on Linux, to a process that still exists, under a Yama - /// policy that is actually restricting. Returns the scope value, or null - /// when nothing here can be named as the refuser. Both the human advice - /// and the durable record read this — a diagnostic that blames the - /// kernel while the record blames the target would be two opinions about - /// one event. + /// The one place that decides whether a restricting policy was + /// OBSERVED: a live attach, on Linux, to a process that still exists, + /// under a Yama policy that is not permissive. Returns the scope value, + /// or null when there is no such policy to name. + /// + /// This proves the policy was IN FORCE, not that it caused the failure + /// in hand — the kernel does not tell the tracer which check rejected + /// it. Both the human advice and the durable record read this one + /// function, and both are worded to that limit, so stderr and the + /// artifact cannot end up holding two opinions about one event. private static string? YamaScope(int pid, bool live) { if (!live || !OperatingSystem.IsLinux()) return null; @@ -186,8 +241,8 @@ private static IEnumerable AttachAdvice(int pid, bool live) return scope == "0" ? null : scope; } - /// The refusing policy, named as the record must name it, or - /// null when no policy can be shown to have refused. + /// The restricting policy in force, named as the record names + /// it, or null when there is none to name. private static string? RefusingPolicy(int pid, bool live) { string? scope = YamaScope(pid, live); @@ -462,7 +517,7 @@ private static int Failed(string[] args, Exception ex) { ["classification"] = ex.GetType().Name, ["detail"] = ex.Message, - ["phase"] = "walk", + ["stage"] = "walk", }, })); return 2; @@ -698,6 +753,29 @@ void CheckRecord(string name, Dictionary doc, Check("witness absent but evaluated is `clean`", $"{Evaluated(false, someScope)["state"]}", "clean"); + // 8. Stage attribution. A permission claim belongs to the one stage a + // permission check applies to. The first cut of this arc gated on + // "did we get a walker", which put every CLR-initialisation + // failure — a target that opened fine and then turned out not to + // be a managed process — under `refused-attach` whenever Yama + // happened to be restricting. + var boom = new InvalidOperationException("the target contains no CLR"); + var afterOpen = ReadFailure(Stage.CreateRuntime, pid: 1, live: true, ex: boom); + Check("a failure after the target opened is never a permission claim", + $"{afterOpen["code"]}", "unreadable-target"); + Check("and it says which stage it fell over at", + $"{afterOpen["stage"]}", "create-runtime"); + if (afterOpen.ContainsKey("policy_in_force")) + fails.Add("create-runtime failure must not cite a ptrace policy"); + + // A dump has no process to trace, so no policy can be in force for + // it — the same call must stay silent about permission there too. + var dumpFail = ReadFailure(Stage.OpenTarget, pid: 0, live: false, ex: boom); + Check("an unreadable dump is not a refusal", + $"{dumpFail["code"]}", "unreadable-target"); + if (dumpFail.ContainsKey("policy_in_force")) + fails.Add("a dump read must not cite a ptrace policy"); + foreach (var f in fails) Console.Error.WriteLine($"FAIL: classifier {f}"); if (fails.Count == 0) diff --git a/docs/runtime-witness-operations.md b/docs/runtime-witness-operations.md index efc929f3..b396afd4 100644 --- a/docs/runtime-witness-operations.md +++ b/docs/runtime-witness-operations.md @@ -56,15 +56,25 @@ Absence has too many preimages to carry meaning, so it is given none: > **Absence of a record means no durable knowledge, never a semantic outcome.** -Every attempted evaluation writes a record when `--out` is given, and the record -states what happened: +Every evaluation **for which persistence was requested** — that is, every run +given `--out` — writes a record, and the record states what happened. A run +without `--out` asked for no durable output and gets none; the witness is a +standalone tool with no other publication point, so `--out` is where the +invariant applies. The record states what happened: | `execution.state` | Exit | Carries | Meaning | | --- | --- | --- | --- | -| `observed` | 1 | `scope`, `verdict`, `retained` | The heap was read; a witness is present. | -| `clean` | 0 | `scope`, `verdict`, `retained` | The heap was read; no witness. | -| `not_evaluated` | 2 | `reason.code`, `reason.detail` | Nothing was read. No verdict is recorded. | -| `error` | 2 | `error.classification` | The heap was readable and the walk broke. | +| `observed` | 1 | `scope`, `retained` | The heap was read; a witness is present. | +| `clean` | 0 | `scope`, `retained` | The heap was read; no witness. | +| `not_evaluated` | 2 | `reason.code`, `reason.stage`, `reason.detail` | Nothing was read. No verdict is recorded. | +| `error` | 2 | `error.classification`, `error.stage` | The heap was readable and the walk broke. | + +`verdict` is **command-specific, not part of the execution contract**: `roots` +carries one (`RETAINED` / `OBSERVED_ONLY` / `ABSENT`), `census` does not, because +"is any of this heap retained at all" is a different question with no such +vocabulary. Inventing a census verdict to make the schema uniform would add a +word nobody measured. What every evaluated state does owe is `scope` and +`retained`. ```jsonc { @@ -72,9 +82,10 @@ states what happened: "execution": { "state": "not_evaluated", "reason": { - "code": "refused-attach", // usage-error | unreadable-target | refused-attach + "code": "refused-attach", // usage-error | unreadable-target | refused-attach + "stage": "open-target", // open-target | create-runtime "detail": "ClrDiagnosticsException: Could not attach to process 4213", - "policy": "kernel.yama.ptrace_scope=1" // only when a refuser can be named + "policy_in_force": "kernel.yama.ptrace_scope=1" } }, "collector": { "tool": "retention-path", "mode": "attach", "target": "4213", … } @@ -88,14 +99,25 @@ record carries no `verdict` key and no `retained` key **at all** — not even `retained: []`, which downstream reads as *"looked, found nothing"* and would re-create the collapse one layer up. -It does not claim a refusal it did not observe. `refused-attach` is a statement -about permission, so it is used only where a refusing policy can be named -(`reason.policy`); every other unreadable target gets the weaker, true -`unreadable-target` with the exception in `detail`. - -`not_evaluated` and `error` are separated by *where* the failure landed: before -the heap was readable, nothing was looked at and the target is not implicated; -after it, the witness itself broke mid-walk and the target is not exonerated. +It does not claim a refusal it did not observe. A run passes through three +stages — **open-target**, **create-runtime**, **walk** — and only the first is +one a permission check applies to. A failure at `create-runtime` means the +target opened and then turned out not to be a readable CLR process, which says +nothing about permission; a failure at `walk` is the witness breaking, not the +target refusing. So `refused-attach` is reserved for `open-target`, and the +ptrace advice on stderr is printed at that stage only — otherwise stderr would +blame the kernel while the record blamed the walk. + +Even there the claim is bounded. `policy_in_force` records that a restricting +Yama policy **was in force**, which is what the collector can see. It does not +record that the policy caused this particular failure, which it cannot: the +kernel does not tell a tracer which check rejected it, and a live process under +`ptrace_scope=1` can fail to open for unrelated reasons. Everything else gets +the weaker, true `unreadable-target` with the exception in `detail`. + +`not_evaluated` and `error` are separated by the same stages: before the heap +was readable, nothing was looked at and the target is not implicated; after it, +the witness broke mid-walk and the target is not exonerated. **A `clean` with no `scope` is malformed, not weak.** A record that does not say what was looked at cannot mean "nothing was there", so consumers must route it From 35e101f7b3a61ee8d13afe362a074c633a300766 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 04:09:01 +0000 Subject: [PATCH 3/3] docs(runtime-witness): say what refused-attach records, not what it proves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The behaviour was already honest — the code is reserved for an open-target failure with a policy observed in force, and `policy_in_force` is worded as an observation — but the comment above it still opened with "a claim about PERMISSION". A later reader would take the enum name as the finding and put the causal claim back by hand. Comment only; no behaviour change. Selftest and build unchanged. Refs #331. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016Lmcv3X9PoELp8CGDfNc9m --- audit/runtime/RetentionPath/Program.cs | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/audit/runtime/RetentionPath/Program.cs b/audit/runtime/RetentionPath/Program.cs index 3d2dfbed..72f149c6 100644 --- a/audit/runtime/RetentionPath/Program.cs +++ b/audit/runtime/RetentionPath/Program.cs @@ -131,14 +131,15 @@ private enum Stage /// Why the heap could not be read, said no more strongly than the /// evidence allows. /// - /// `refused-attach` is a claim about PERMISSION, so it is reserved for a - /// failure at the one stage a permission check applies to, with a - /// restricting policy actually in force. Even then the policy is recorded - /// as policy_in_force, not as the proven cause: a live process - /// under `ptrace_scope=1` can fail to open for reasons that have nothing - /// to do with Yama, and this collector cannot tell those apart. Naming - /// what was in force is observation; naming it as the refuser would be - /// the same unearned confidence the execution record exists to prevent. + /// `refused-attach` records a live attach that failed at `open-target` + /// while a restrictive policy was observed in force. It does not identify + /// the cause. A live process under `ptrace_scope=1` can fail to open for + /// reasons that have nothing to do with Yama, and the kernel does not + /// tell a tracer which check rejected it, so the code names a situation + /// and policy_in_force names what was seen — neither names a + /// culprit. Reading the code as a proven permission failure would put + /// back the same unearned confidence the execution record exists to + /// prevent: the enum is not smarter than the evidence behind it. /// /// Everything else — including a target that opened and then turned out /// not to be a readable CLR process — gets the weaker, true