diff --git a/audit/runtime/README.md b/audit/runtime/README.md index cb4ebb9f..2a14dea5 100644 --- a/audit/runtime/README.md +++ b/audit/runtime/README.md @@ -44,6 +44,10 @@ audit/runtime/ PropertyChangedStorm/ # C# PropertyChanged-storm profiler — Windows/build-required, NOT CI-gated PropertyChangedStorm.csproj # net472; Microsoft.Diagnostics.Tracing.TraceEvent Program.cs # TraceEvent over an .etl: per-property raise frequency, storm findings + RetentionPath/ # C# retention witness — net8.0, CROSS-PLATFORM, build smoked in CI + RetentionPath.csproj # net8.0; Microsoft.Diagnostics.Runtime (ClrMD 3.x) + Newtonsoft.Json + Program.cs # census / roots verbs, exit-code tiers, runtime.json writer + Heap.cs # mark-from-roots, BFS root->object paths with field names ``` ## How the leak-harness works (Plan.md §4.1) @@ -121,6 +125,105 @@ python audit/runtime/ingest.py --propertychanged-storm \ # a located storm clusters with a static INPC0xx in the same file. ``` +## Retention paths — is it retained, and by whom (Plan.md §4) + +The stack table above has promised *"Heap analysis (retained, duplicates, **retention +paths**) — ClrMD"* since Plan.md §4. `RetentionPath/` is that half. `HeapCounter` +counts instances of named types; this answers the two questions that actually decide +a leak hunt — *is any of it retained at all*, and *who is holding it*. + +Unlike its three neighbours it is **net8.0 and cross-platform**, needs no procdump, +and can attach to a live PID. CI builds it and smokes its usage surface on every +push; the end-to-end demo runs on Linux and the WPF pair on Windows. + +### `census` — is there anything to hunt? + +```console +$ retention-path census --pid 1234 --out runtime.json + +roots : 308 objects +on the heap : 4 270 155 objects 573 MB +REACHABLE from roots : 4 144 653 objects 403 MB +uncollected garbage : 125 502 objects 170 MB +>>> 70,4% of the heap is genuinely RETAINED — something holds it; run `roots` +``` + +**This distinction is not pedantry.** `ClrHeap.EnumerateObjects()` walks the heap +segments linearly and returns *everything allocated, including garbage the GC has +not collected yet*. A big heap is not evidence of a leak. `HeapCounter` mitigates +that by forcing a GC in the target first, which works when you can drive the target; +marking from the roots answers it directly and needs no cooperation. If the retained +share is low, **stop** — there is no reference to hunt, and the next question is +about GC timing, not about who holds what. + +### `roots` — what holds the TYPICAL instance + +```console +$ retention-path roots --pid 1234 --type GTDGoody + +verdict: RETAINED — BrokerDataClasses.GTDGoody: 130 000 on the heap, 129 903 reachable … + +#1 25/50 resolved (50,0%) — via [static-event], 7 hops + [PinnedHandle] System.Object[] + BrokerDataClasses.GTD + BrokerDataClasses.CalcProcentGTD (.k__BackingField) + BrokerDataClasses.GTDGoody (.fMainObject) +``` + +The field names come from ClrMD's `EnumerateReferencesWithFields`; they are what turn +*"this object is alive"* into *"**this field** is holding it"* — the sentence a +developer can act on. A **delegate hop** in the path is what makes it a static +*event* rather than a plain static field, which is the distinction OwnAudit's +`correlate.py` keys its `high` tier on. + +**Why it samples, and what the percentages mean.** *"Who holds this object"* is +ill-posed for an object reachable from many roots: there are as many answers as +there are paths, and the shortest is an arbitrary pick rather than an explanation. +So the walk resolves a SAMPLE of paths and ranks them as a histogram — the retainer +accounting for 129 900 of 130 000 is the leak; the three hanging off the stack are +noise. The shares are shares **of the resolved sample**, never of the population. +The verdict is not sampled: reachability and the durable/transient census run over +every instance, so no display budget (`--sample`, `--max-hops`) can change the +diagnosis or the exit code. + +### What it does not do — read this before trusting it + +- **No dominator tree.** The principled form of "who holds it" is dominance: which + single reference, if cut, makes the object collectable, and how much that frees. + It also answers honestly when two references hold an object jointly, naming the + point where the paths meet instead of picking one. That is what Eclipse MAT and + dotMemory are built on. The A3 witness was extracted from PR #280 **without** it, + deliberately; Own.NET#334 records what the implementation argued. The sampled + histogram is a weaker instrument, and the ranking is how it stays honest about it. +- **A `[stack]` root is not retention** — the object is live in a frame right now, + and is labelled so it is not mistaken for a leak. Same for `[finalizer]`. A + verdict of `RETAINED` requires at least one *durable* retainer. +- **It matches the type, not the type's spelling.** `--type GTDGoody` must not match + `System.Func<…GTDGoody…>` — a cached lambda whose generic *argument* mentions it. + It did, during development, and confidently reported a 2-hop path to the wrong + object. +- **Attaching suspends the target.** On a multi-GB heap the mark pass is minutes, + not seconds — take a dump. + +### Exit codes, and why exit 2 exists + +| Exit | Meaning | +| --- | --- | +| 0 | The heap was read. `ABSENT` / `OBSERVED_ONLY` — nothing durably retains the type. | +| 1 | The heap was read. `RETAINED` — a durable path exists, and it is printed. | +| 2 | **The heap was not read.** Usage error, unreadable target, refused attach. | + +*Not looking* and *looking and finding nothing* are different outcomes, and +collapsing them is how a monitoring pipeline learns to report health it never +measured. Permissions, the Yama `ptrace_scope` cases and the CI rules are in +[`docs/runtime-witness-operations.md`](../../docs/runtime-witness-operations.md). + +Output is the `runtime.json` contract (`OwnAudit/docs/runtime-contract.md`), so +`OwnAudit/runtime/correlate.py` consumes it with no adapter — giving the three-way +split its missing input: **confirmed** (a static leak finding whose type also shows +up retained), **static-only** (probable FP), and **runtime-only** — retention with +nothing static to explain it, which is a rule request rather than a report. + ## Selftest `ingest.py` carries embedded-fixture selftests (no harness, no Windows needed) and @@ -138,8 +241,12 @@ python audit/runtime/ingest.py --selftest the duplicate detector and the PropertyChanged-storm profiler), the leak-harness scenario schema + one scenario, runtime rule mappings in the taxonomy (categories 2/3/4/6/11), the C# leak-harness skeleton, the C# duplicate-immutable detector - (strings), and the C# PropertyChanged-storm profiler (ETW). -- **Deferred:** duplicate detection for arbitrary immutable types (field-by-field + (strings), the C# PropertyChanged-storm profiler (ETW), and the **retention + witness** (`RetentionPath/`: census + ranked root paths, `runtime.json`, exit-code + tiers, CI-gated build and demo). +- **Deferred:** the **dominator tree / retained sizes** for the retention witness + (Own.NET#334 — the A3 extraction left it out on purpose; the sampled histogram is + what ships), duplicate detection for arbitrary immutable types (field-by-field content equality), the diagnostic-build INPC `EventSource` instrumentation in the target + PerfView/SematixTrace capture wiring, and a scenario corpus for the top-N screens. diff --git a/audit/runtime/RetentionPath/Heap.cs b/audit/runtime/RetentionPath/Heap.cs index 67f71c12..38b6a13b 100644 --- a/audit/runtime/RetentionPath/Heap.cs +++ b/audit/runtime/RetentionPath/Heap.cs @@ -31,7 +31,10 @@ namespace OwnNet.Audit.Runtime /// is how a leak hunt goes wrong. /// /// The principled version of this is a dominator tree (which single reference, if cut, - /// frees the object — and how much memory that frees). See the README. + /// frees the object — and how much memory that frees). It is NOT implemented here: the + /// A3 witness was extracted from PR #280 without it, deliberately. See + /// `audit/runtime/README.md` § "Retention paths" for what the sampled histogram does and + /// does not answer, and Own.NET#334 for what the dominator implementation argued. /// internal sealed class RetentionWalker : IDisposable { @@ -347,11 +350,6 @@ internal static bool IsType(string heapType, string wanted) return string.Equals(simple, wanted, StringComparison.Ordinal); } - /// - /// The dominator tree of the whole live graph, with retained sizes. This is the well-posed - /// version of "who holds it": not a path, but the one reference whose removal frees the object. - /// - public void Dispose() { _runtime.Dispose();