From 555c565d8a99d8922eeb0015a9fe2e7aefe184ca Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 07:38:27 +0000 Subject: [PATCH 1/6] docs(notes): assess Leaf/DMIR against what OwnAudit actually has An outside reading of arXiv:2607.15025 proposed Leaf as a concrete building block for OwnAudit (8.5/10) with a separate Rust-analysis direction (9/10) and five ideas to adopt. The paper is real and accurately summarised; the proposal does not survive contact with the repositories. Leaf instruments Rust MIR. OwnAudit audits .NET 4.7.2 / WPF -- 98 sarif, 70 py, 31 cs, zero .rs, no Cargo.toml, and a runtime half that is a ClrMD heap walk emitting ownAudit/runtime/v1. There is no MIR in .NET and Leaf cannot observe a CLR process, so the proposed adapter pipeline has no endpoints. The Rust-analysis direction is further off: P-017 targets OwnTS and OwnJVM, and the project's thesis is ownership discipline for a GC language that lacks it, which Rust is not. The likely confusion is P-022 -- the analyzer written in Rust, not analyzing Rust. Of the five ideas: static-site/runtime-fact correlation and explicit evidence gaps are already shipped, the oracle pattern is already shipped, the provider-agnostic seam already exists as a versioned runtime.json schema, and static-directed targeted instrumentation solves a per-event cost problem a post-scenario heap snapshot does not have. The sharpest inversion is the evidence-gap item, offered as something to import when it is already the CI-proven exit-code contract in runtime-witness-operations.md. Recorded so the proposal is not re-derived, same purpose as P-034's do-not-re-derive table. The one genuine takeaway is about our own roadmap: the runtime layer is single-purpose today while Plan.md schedules two more consumers (storm frequency, duplicate heap data), so the shared-schema-vs-bespoke-collector question is worth settling before the second collector exists. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/leaf-dmir-assessment.md | 91 ++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 docs/notes/leaf-dmir-assessment.md diff --git a/docs/notes/leaf-dmir-assessment.md b/docs/notes/leaf-dmir-assessment.md new file mode 100644 index 00000000..3e740d10 --- /dev/null +++ b/docs/notes/leaf-dmir-assessment.md @@ -0,0 +1,91 @@ +# Leaf / DMIR — assessment against what OwnAudit actually has + +Working note. **Trigger:** an outside reading of *"Leaf: An Instrumentation-based +Dynamic Analysis Framework for Rust"* +([arXiv:2607.15025](https://arxiv.org/abs/2607.15025), Omidvar Tehrani, Gaboardi, +Sumner, Ko) proposed it as "a concrete building block for OwnAudit" at 8.5/10, +with a separate Rust-analysis direction at 9/10, and listed five ideas to adopt. + +The paper is real and summarised accurately: Leaf instruments **Rust MIR** via +`rustc` internals and delivers **DMIR**, an event-driven stream carrying static +MIR identity plus runtime facts, with a concolic executor / sanitizer / tracer +built on top. The architecture is genuinely good. + +The proposal does not survive contact with the repositories. Recorded here so +nobody re-derives it — same purpose as +[P-034](../proposals/P-034-runtime-lifetime-guard.md)'s "do not re-derive" table. + +## The blocking fact: wrong runtime + +**Leaf instruments Rust MIR. OwnAudit audits .NET.** + +Verified against `PhysShell/OwnAudit` at clone time: + +- README: an orchestrator over "a legacy **.NET 4.7.2 / WPF / DevExpress** app". +- File census: 98 `.sarif`, 70 `.py`, 46 `.json`, 38 `.md`, 31 `.cs` — and + **zero `.rs`, no `Cargo.toml`**. +- The runtime half is a **ClrMD heap walk** over a CLR process, emitting + `runtime.json` (`schema: "ownAudit/runtime/v1"`). + +There is no MIR in .NET and Leaf cannot observe a CLR process. The proposed +`Leaf → adapter → RuntimeEvidenceEvent → OwnAudit rules` pipeline is not +buildable at any effort level. The 8.5/10 is for a wire that has no endpoints. + +**The 9/10 is further off still.** [P-017](../proposals/P-017-multi-stack-frontends.md) +takes the core beyond .NET to **OwnTS** and **OwnJVM (Java/Kotlin)** — Rust is +not a target and structurally should not be: this project's entire thesis is +bringing ownership discipline to a **GC language that lacks it**, and the +ROADMAP carries a standing section on *why not a Rust-style borrow checker for +C#*. Rust already has the borrow checker; there is no gap to sell into. + +The likely source of the error is real and easy to trip on: **Own.NET's core is +being migrated *to* Rust** ([P-022](../proposals/P-022-rust-core-migration.md)). +That is the analyzer *written in* Rust, not the analyzer *analyzing* Rust. + +## The five "adopt these", scored against the tree + +| # | Proposed idea | Reality | +|---|---|---| +| 1 | Provider-agnostic Runtime Evidence IR, so we don't become a shell over Leaf | **Seam already exists.** `runtime.json` is a versioned schema consumed by `runtime/cli.py`; the collector is already swappable. What differs is snapshot vs event stream — a real question, but not this one | +| 2 | Correlate static site with runtime fact | **Already shipped.** Member-aware matching (2026-07-19): a `subscription-leak` finding's canonical `event 'A.B.Holder.Member'` identity must equal a retention root's `(short(holder), member)`; the matched root is reported as `root_holder`/`root_member`. Evidence in `docs/evidence/gtd-runtime-transition.md` | +| 3 | Static pass drives targeted instrumentation | **Not built — and the motivation does not transfer.** Leaf needs targeting because per-event MIR instrumentation is reportedly 19×–396×. A post-scenario heap snapshot has no per-event cost to target down. Only becomes relevant *if* OwnAudit moves to event-stream instrumentation, which is a decision on its own merits | +| 4 | Explicit evidence gaps: `COMPLETE / PARTIAL / OPAQUE / UNSUPPORTED` | **Already a machine contract, and stronger.** [`runtime-witness-operations.md`](../runtime-witness-operations.md): exit 0 = heap read, nothing retained; exit 1 = read, retained; **exit 2 = the heap was not read**. A refused attach writes **no** `runtime.json` — there is no verdict to record. *"Proven by CI: a denied attach exits 2, names the policy that refused, and leaves no artifact behind."* Plus `NO-TOOL: skipped` coverage maps, the OWN050/051/052 advisories, and P-036's rule that a check "may not pretend the call was proven harmless" | +| 5 | Use it as an oracle for static rules | **Pattern already shipped**, minus Leaf. The 3-way oracle (Own.NET vs Infer# vs CodeQL) plus the runtime buckets: **confirmed** (gate on these), **static-only** (suspect FP or unexercised path), **runtime-only** = *"the analyzer's blind spot — candidate for a new rule"* | + +So: two already shipped, one shipped as a pattern, one already has its seam, and +one solves a cost problem this stack does not have. + +The sharpest inversion is #4. The proposal offered explicit evidence gaps as +something to import — *"сделанная частью машинного контракта, а не красивой +фразой в README"*. It is already the machine contract, it is CI-proven, and +distinguishing *not looking* from *looking and finding nothing* is one of the +oldest rules in this codebase. + +## What is actually worth taking + +One thing, and it is about our own roadmap rather than about Leaf. + +Leaf's real contribution is **decoupling**: one semantic runtime layer, many +analyzers above it, instead of a single-purpose sanitizer that then spends +twenty years impersonating infrastructure. OwnAudit's runtime layer is today +**single-purpose** — retention/leak confirmation. But [`Plan.md`](../../Plan.md) +§2 already schedules two more runtime consumers: **category 6** (PropertyChanged +storms / frequency) and **category 11** (duplicate immutable data on the heap, +flagged there as *"the project's gold"*). + +That makes the live question: **one shared runtime-evidence schema, or three +bespoke collectors?** Worth answering *before* the second collector is written, +not after. That is a genuine, correctly-scoped takeaway — and note it is +answerable entirely inside .NET, with ClrMD and ETW, with no Rust anywhere near +it. + +## Provenance and limits + +- The paper's existence, title, authors and subject were verified against arXiv. +- The **overhead figures** (19×–396×, ~3.5× for concrete types, ~2.8× for raw + addresses, ~50 DMIR callback types, the `nightly-2026-07-01` pin) come from + the summary being assessed and were **not** independently checked against the + paper or the repository. They are quoted above only to show that the + motivation for idea #3 is cost, not to rely on the magnitudes. +- OwnAudit was read at a shallow clone of `main`; claims are about that tree. +- **Recorded, not scheduled.** No work item is filed off this note. From 8df84a9d42bdd0ebb8d18d91adb2704c3d014f75 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 11:57:33 +0000 Subject: [PATCH 2/6] docs(notes): the runtime-substrate question is overdue by two consumers, not pending Follow-up on the Leaf assessment after the original author accepted the correction and refined the takeaway. Two changes, both from reading the tree rather than from the paper. Accepts the second concept-level takeaway: separate the collection substrate from the analysis-facing representation. Leaf does not hand probes to analyzers -- probes are shaped for cheap instrumentation and an adapter reconstructs structured events -- and that boundary is what stops a snapshot and an event stream being forced into one datatype. Replaces "decide before the second dynamic consumer" with what is actually there. audit/runtime/ already ships four collectors, so Plan.md categories 6 and 11 exist as code with bespoke JSON each. Two producers of runtime.json emit two different schema strings -- RetentionPath says own-runtime/1, the newer stand-side collector and docs and every fixture say ownAudit/runtime/v1 -- a lift-out artifact, but live. Nothing enforces either: correlate.py reads only retained/type/count/expected/bytes/roots/kind/holder/member and never touches schema, scenario, iterations, collector or verdict, and the one assertion checks a producer against itself. No impact today since the correlator ignores unknown fields; the risk is latent and cheap to settle now. Adds a caution the proposed hybrid needs: moving completeness into the envelope collides with the existing fail-closed contract, where a refusal means exit 2 and no artifact at all. A completeness: PARTIAL field would create the first valid-but-admittedly-incomplete artifact -- possibly right for an event trace, but a weakening to decide deliberately, not inherit. Records the invariant that caused the original error, since it will recur: do not infer analyzed-language support from the implementation language of the core. P-022 moves the analyzer's implementation to Rust; it does not change the audited runtime or make Rust an analysis target. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/leaf-dmir-assessment.md | 119 ++++++++++++++++++++++++----- 1 file changed, 99 insertions(+), 20 deletions(-) diff --git a/docs/notes/leaf-dmir-assessment.md b/docs/notes/leaf-dmir-assessment.md index 3e740d10..eca24ce2 100644 --- a/docs/notes/leaf-dmir-assessment.md +++ b/docs/notes/leaf-dmir-assessment.md @@ -63,29 +63,108 @@ oldest rules in this codebase. ## What is actually worth taking -One thing, and it is about our own roadmap rather than about Leaf. - -Leaf's real contribution is **decoupling**: one semantic runtime layer, many -analyzers above it, instead of a single-purpose sanitizer that then spends -twenty years impersonating infrastructure. OwnAudit's runtime layer is today -**single-purpose** — retention/leak confirmation. But [`Plan.md`](../../Plan.md) -§2 already schedules two more runtime consumers: **category 6** (PropertyChanged -storms / frequency) and **category 11** (duplicate immutable data on the heap, -flagged there as *"the project's gold"*). - -That makes the live question: **one shared runtime-evidence schema, or three -bespoke collectors?** Worth answering *before* the second collector is written, -not after. That is a genuine, correctly-scoped takeaway — and note it is -answerable entirely inside .NET, with ClrMD and ETW, with no Rust anywhere near -it. +Two things, both concept-level, neither depending on Leaf. + +**(a) One runtime substrate, many analyzers.** Leaf's real contribution is +decoupling: instrumentation exists separately from any particular analysis, so +one semantic layer (50 DMIR callbacks) serves a concolic executor, a sanitizer +and a tracer. The boundary transfers; the MIR does not. + +**(b) Separate the *collection* substrate from the *analysis-facing* +representation.** Subtler than "one shared schema", and the part worth stealing +outright. Leaf does not hand probes to analyzers: probes are shaped for cheap +instrumentation, and a Probe-to-DMIR adapter reconstructs structured events for +analysis. The two formats are deliberately different. The failure this avoids is +one universal envelope that everything gets poured into *because the schema +already exists* — a snapshot and an event stream should not be forced to +pretend they are one datatype. + +### Measured against the tree, the question is already overdue + +The natural framing — "decide before the second dynamic consumer" — is **too +late by two consumers**. `Own.NET/audit/runtime/` already contains four C# +collectors: + +| collector | output | schema field | +|---|---|---| +| `RetentionPath` | `runtime.json` | `"own-runtime/1"` | +| `OwnAudit/src/OwnAudit.Runtime` (`RuntimeReport.cs`) | `runtime.json` | `"ownAudit/runtime/v1"` | +| `LeakHarness` | `artifacts/own-audit/leak-harness.json` | none | +| `DuplicateDetector` | `artifacts/own-audit/duplicate-detector.json` | none | +| `PropertyChangedStorm` | `artifacts/own-audit/propertychanged-storm.json` | none | + +So [`Plan.md`](../../Plan.md) categories 6 (PropertyChanged storms) and 11 +(duplicate immutable heap data, flagged there as *"the project's gold"*) are not +future work to design around — they exist as code, each with its own bespoke +JSON. Three findings follow, all checkable: + +1. **Two producers of `runtime.json` emit two different schema strings.** + `RetentionPath` (both of its writers) says `own-runtime/1`; the newer + stand-side collector, `docs/runtime-contract.md`, and every fixture say + `ownAudit/runtime/v1`. This is a lift-out artifact — the collector plan + (2026-07-18) ports `stackpeek` into `OwnAudit.Runtime` while the older + `RetentionPath` stays canonical in Own.NET — not sloppiness. But it is live. +2. **Nothing enforces either.** `runtime/correlate.py` consumes exactly + `retained / type / count / expected / bytes / roots / kind / holder / + member`. It never reads `schema`, `scenario`, `iterations`, `collector`, or + `verdict`. The only assertion on the string is + `HeapCollectorContractTests.cs:83`, and it checks the producer against + itself. A `runtime.json` with the wrong version — or none — correlates + silently. +3. **The envelope fields are decorative.** `scenario` and `iterations` are + documented and present in fixtures but unread; `collector` (a genuine + provenance fingerprint) and `verdict` are emitted by `RetentionPath` but + appear in neither the documented schema nor the consumer. + +**No impact today** — the correlator is deliberately tolerant of unknown fields, +which is a reasonable forward-compat choice. The risk is latent and the fix is +cheap *now*, which is exactly the argument for settling it before a fourth +consumer inherits the ambiguity. + +### On the proposed shape + +The hybrid — a shared envelope carrying capture/process/runtime/scenario +identity, collector fingerprint, config digest and provenance, with +**typed-distinct** `HeapSnapshotEvidence` vs `EventTraceEvidence` payloads — is +the right instinct, and finding (b) above is why: snapshot and trace genuinely +are different datatypes. + +One caution, from finding (2) rather than from taste. Putting +**completeness/refusal into the envelope** collides with the contract we already +have. Today the rule is binary and out-of-band: exit 2, and **no artifact at +all** — *"there is no verdict to record"*. Adding a `completeness: PARTIAL` +field creates, for the first time, a valid artifact that is admittedly +incomplete. That may well be worth it for an event trace, where partial capture +is normal rather than exceptional. But it is a **weakening of a fail-closed +invariant**, not a free addition, and it should be decided deliberately — +per-payload-family, most likely — rather than inherited from a Rust paper whose +gaps are a property of selective instrumentation we do not do. + +## The invariant worth stating separately + +The error that produced the original proposal is natural, will recur, and is +cheap to inoculate against: + +> **Do not infer analyzed-language support from the implementation language of +> the Own.NET core.** P-022 moves the *analyzer's implementation* to Rust. It +> does not change the audited runtime, does not introduce Rust MIR, and does not +> make Rust an analysis target — P-017's targets are OwnTS and OwnJVM. + +Implementation language and analyzed language are orthogonal axes. Conflating +them is what turned a Rust-only instrumentation framework into an 8.5/10 +recommendation for a .NET auditor. ## Provenance and limits - The paper's existence, title, authors and subject were verified against arXiv. - The **overhead figures** (19×–396×, ~3.5× for concrete types, ~2.8× for raw - addresses, ~50 DMIR callback types, the `nightly-2026-07-01` pin) come from - the summary being assessed and were **not** independently checked against the - paper or the repository. They are quoted above only to show that the - motivation for idea #3 is cost, not to rely on the magnitudes. -- OwnAudit was read at a shallow clone of `main`; claims are about that tree. + addresses, ~50 DMIR callback types, the `nightly-2026-07-01` pin, and the + three demo analyses) were **not** checked here against the paper. The author + of the original summary re-verified them against it afterwards and reports + them correct; that is second-hand for this note's purposes, and they are + quoted only to show that the motivation for idea #3 is cost, not to rely on + the magnitudes. +- OwnAudit was read at a shallow clone of `main`; the collector inventory and + schema-string findings are about that tree plus `Own.NET/audit/runtime/` at + this branch's base commit. - **Recorded, not scheduled.** No work item is filed off this note. From 6607ec14d5b644908d3e75ce40e336ce2344cbff Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 12:41:51 +0000 Subject: [PATCH 3/6] docs(notes): separate acquisition from interpretation; version artifacts before unifying them Two corrections and a resequencing, all verified against the tree. Counting fix: the note said four C# collectors under Own.NET/audit/runtime and then showed a five-row table, because the fifth row is the lift-out producer in the other repository. There are four directories under audit/runtime and five producer implementations across the two repos. Terminology fix, and it is load-bearing rather than pedantic. "Collector" conflated two layers. Checked by the APIs each one calls: PropertyChangedStorm opens an ETWTraceEventSource over an .etl captured externally by PerfView/xperf/logman and acquires nothing at all; DuplicateDetector and LeakHarness interpret a .dmp and can shell out to procdump; RetentionPath does AttachToProcess or LoadDump. So the layering is acquisition (live attach / .dmp / .etl) then interpretation then analysis-facing artifact. That sharpens the takeaway: the acquisition representation should not be the evidence representation, and what deserves sharing is a runtime artifact contract, not a collection substrate. A .dmp and an .etl are already good acquisition substrates; unifying them would unify the wrong layer. Resequencing: a shared envelope is the right destination but the wrong first move, since designing commonality up front is the mistake Leaf avoids. The minimal work is per-family mandatory schema identity, then readers that validate family and major version while staying tolerant of unknown fields, and only then an envelope extracted from commonality proven by shipped schemas. Step two is the actual gap: correlate() duck-types any dict carrying a retained array. Splits completeness into collection outcome, artifact validity and evidence coverage. Refusal still means exit 2 and no artifact; validity is the missing layer; coverage belongs to the artifact family, so heap retention can stay total-or-no-artifact while an ETW trace carries real bounds. Records the invariant that artifact existence proves execution of the producer contract, not completeness of observation. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/leaf-dmir-assessment.md | 121 ++++++++++++++++++++++------- 1 file changed, 93 insertions(+), 28 deletions(-) diff --git a/docs/notes/leaf-dmir-assessment.md b/docs/notes/leaf-dmir-assessment.md index eca24ce2..006c780b 100644 --- a/docs/notes/leaf-dmir-assessment.md +++ b/docs/notes/leaf-dmir-assessment.md @@ -82,16 +82,35 @@ pretend they are one datatype. ### Measured against the tree, the question is already overdue The natural framing — "decide before the second dynamic consumer" — is **too -late by two consumers**. `Own.NET/audit/runtime/` already contains four C# -collectors: - -| collector | output | schema field | -|---|---|---| -| `RetentionPath` | `runtime.json` | `"own-runtime/1"` | -| `OwnAudit/src/OwnAudit.Runtime` (`RuntimeReport.cs`) | `runtime.json` | `"ownAudit/runtime/v1"` | -| `LeakHarness` | `artifacts/own-audit/leak-harness.json` | none | -| `DuplicateDetector` | `artifacts/own-audit/duplicate-detector.json` | none | -| `PropertyChangedStorm` | `artifacts/own-audit/propertychanged-storm.json` | none | +late by two consumers**. There are **five runtime-evidence producer +implementations across the two repositories**: four under +`Own.NET/audit/runtime/`, plus the lift-out producer in +`OwnAudit/src/OwnAudit.Runtime`. + +"Collector" is the wrong word for them, and the distinction is load-bearing +rather than pedantic — it decides which layer a new contract belongs to. +Verified by the APIs each one actually calls: + +| runtime evidence producer | acquisition it performs | material it interprets | output | schema field | +|---|---|---|---|---| +| `RetentionPath` | `AttachToProcess`, `procdump` | live CLR **or** `.dmp` | `runtime.json` | `"own-runtime/1"` | +| `OwnAudit.Runtime` (`RuntimeReport.cs`) | stand-side collector | live heap | `runtime.json` | `"ownAudit/runtime/v1"` | +| `LeakHarness` | `procdump` (orchestrates the scenario) | `.dmp` | `leak-harness.json` | none | +| `DuplicateDetector` | `procdump` (optional) | `.dmp` | `duplicate-detector.json` | none | +| `PropertyChangedStorm` | **none** | `.etl` captured externally | `propertychanged-storm.json` | none | + +`PropertyChangedStorm` is the clean case: it opens an `ETWTraceEventSource` over +an `.etl` that PerfView/xperf/logman captured against the target's +`OwnNet-Sematix-INPC` EventSource. It acquires nothing. So the real layering is +**acquisition** (live attach / `.dmp` / `.etl`) → **interpretation** → +**analysis-facing artifact** → `correlate.py` / `ingest.py` → SARIF. + +That reframes takeaway (b) more sharply than "one substrate, many analyzers": +*the acquisition representation need not — and should not — be the evidence +representation.* A `.dmp` and an `.etl` are already good acquisition substrates +for their jobs; turning them into a universal runtime-event JSON would unify the +wrong layer. The thing worth sharing is a **runtime artifact contract**, not a +collection substrate. So [`Plan.md`](../../Plan.md) categories 6 (PropertyChanged storms) and 11 (duplicate immutable heap data, flagged there as *"the project's gold"*) are not @@ -121,24 +140,70 @@ which is a reasonable forward-compat choice. The risk is latent and the fix is cheap *now*, which is exactly the argument for settling it before a fourth consumer inherits the ambiguity. -### On the proposed shape - -The hybrid — a shared envelope carrying capture/process/runtime/scenario -identity, collector fingerprint, config digest and provenance, with -**typed-distinct** `HeapSnapshotEvidence` vs `EventTraceEvidence` payloads — is -the right instinct, and finding (b) above is why: snapshot and trace genuinely -are different datatypes. - -One caution, from finding (2) rather than from taste. Putting -**completeness/refusal into the envelope** collides with the contract we already -have. Today the rule is binary and out-of-band: exit 2, and **no artifact at -all** — *"there is no verdict to record"*. Adding a `completeness: PARTIAL` -field creates, for the first time, a valid artifact that is admittedly -incomplete. That may well be worth it for an event trace, where partial capture -is normal rather than exceptional. But it is a **weakening of a fail-closed -invariant**, not a free addition, and it should be decided deliberately — -per-payload-family, most likely — rather than inherited from a Rust paper whose -gaps are a property of selective instrumentation we do not do. +### The cheap next step is versioning, not unification + +A shared envelope carrying capture/process/scenario identity, producer +fingerprint, config digest and provenance, with typed-distinct payloads, is the +right *destination*. It is the wrong *first move*: it designs commonality up +front, which is precisely the mistake Leaf avoids. Leaf's lesson is **define the +boundary first, derive the representation second**. + +So the minimal work that closes the real hole found above, in order: + +1. **Every analysis-facing artifact gets its own mandatory schema identity** — + one per family, e.g. `heap-retention`, `leak-growth`, + `duplicate-immutable`, `propertychanged-storm` (names illustrative, not + normative). Today four of the five producers emit no version at all. +2. **Readers must validate family and major version.** Tolerant of unknown + *fields*, intolerant of unknown *identity*: + + | input | verdict | + |---|---| + | known family + known major + unknown extra fields | accept | + | missing schema | reject | + | wrong artifact family | reject | + | unknown major | reject | + + This is the actual gap. `correlate()` currently takes a `dict` and + structurally duck-types it on `retained`/`type`/`count`/… — so *any* JSON + containing a `retained` array is treated as the contract. +3. **Only then extract the envelope**, from commonality *proven* by four + shipped schemas rather than drawn in advance. Fields have to earn the right + to be called common. + +This closes the found defect now, changes no fail-closed semantics, and commits +to no universal-envelope architecture. + +### Split `completeness` into three orthogonal things + +The caution raised earlier resolves cleanly once the word is broken up. These +are three different states and are currently drifting toward one label: + +- **Collection outcome.** Success → an artifact may exist. Refusal or failure → + exit 2 and the artifact **must not** exist. *The existing fail-closed + invariant, unchanged.* +- **Artifact validity.** Given an artifact: known schema, required fields, + well-formed, producer contract satisfied → valid, else reject. *This is the + layer that is missing today* (step 2 above). +- **Evidence coverage.** Belongs to the **artifact family**, not to a global + enum. For heap retention the family contract can simply be + total-or-no-artifact — no `PARTIAL` ever. For an ETW trace the physics differ + and coverage is real data: capture interval, events lost, buffers lost, trace + truncated, provider-enabled interval. A valid storm artifact can legitimately + rest on a bounded trace if the claim it makes is bounded accordingly. + +That keeps "refusal → no artifact" intact, because *collector refused* and +*collector succeeded over bounded observation* are different states, not two +shades of one. The invariant to write down: + +> **Artifact existence proves successful execution of its producer contract, +> not completeness of every possible observation. Coverage semantics belong to +> the artifact family and must never be inferred from artifact existence +> alone.** + +This is strictly better than a global `"completeness": "PARTIAL"`, which within +a year means five different things per producer and then grows a +`PARTIAL_BUT_USABLE`. ## The invariant worth stating separately From 23c16d28d698ac252ef07c88c241e43db1bf5594 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 14:04:05 +0000 Subject: [PATCH 4/6] docs(tasks): file the runtime artifact identity defect as a scoped spec Fixes two errors in the assessment note and promotes the one finding that earned it from recorded to a written work item. Arithmetic fix: the note claimed four of five producers emit no version, which contradicted its own table. Ground truth -- RetentionPath emits a schema string twice, OwnAudit.Runtime once, and LeakHarness, DuplicateDetector and PropertyChangedStorm emit none. Three of five, not four. Recounts by artifact family instead, since families are what would be versioned: five producer implementations emit four analysis-facing families, and all four have an identity problem -- heap-retention carries two conflicting identities, the other three carry none. Also drops a stale tail that still spoke of a fourth consumer inheriting the ambiguity; four families and five producers already exist, so the trigger is the artifact surface expanding further. Adds docs/tasks/runtime-artifact-identity.md in the established task-spec format. Scope is held deliberately small: one identity per shipped family, an explicit compatibility decision for the heap-retention lift-out split, readers that reject missing identity, wrong family and unsupported major while still accepting unknown fields, and negative fixtures for every rejection case. The load-bearing acceptance case is that a valid-looking retained[] payload with a wrong or missing schema must not correlate -- without it the change is a decorative equality check. A common envelope, any completeness enum, acquisition changes and SARIF changes are all explicitly out of scope, with a guard telling implementers to stop if they start designing a field every family would share. Priority recorded as small correctness debt with high leverage: not a blocker, sequenced after the boundary and gate work already in flight. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/leaf-dmir-assessment.md | 30 +++++- docs/tasks/runtime-artifact-identity.md | 132 ++++++++++++++++++++++++ 2 files changed, 158 insertions(+), 4 deletions(-) create mode 100644 docs/tasks/runtime-artifact-identity.md diff --git a/docs/notes/leaf-dmir-assessment.md b/docs/notes/leaf-dmir-assessment.md index 006c780b..eb08b80e 100644 --- a/docs/notes/leaf-dmir-assessment.md +++ b/docs/notes/leaf-dmir-assessment.md @@ -137,8 +137,24 @@ JSON. Three findings follow, all checkable: **No impact today** — the correlator is deliberately tolerant of unknown fields, which is a reasonable forward-compat choice. The risk is latent and the fix is -cheap *now*, which is exactly the argument for settling it before a fourth -consumer inherits the ambiguity. +cheap *now*, which is exactly the argument for settling it before the runtime +artifact surface expands further. + +### Counted by artifact family, which is what would get versioned + +Producers are the wrong unit — five implementations emit **four** analysis-facing +artifact families, and it is the families that need identity: + +| artifact family | producer(s) | identity status | +|---|---|---| +| heap-retention | `RetentionPath` + `OwnAudit.Runtime` | **two conflicting identities** (`own-runtime/1` vs `ownAudit/runtime/v1`) | +| leak-growth | `LeakHarness` | **none** | +| duplicate-immutable | `DuplicateDetector` | **none** | +| propertychanged-storm | `PropertyChangedStorm` | **none** | + +> **All four shipped runtime artifact families have an identity problem: +> heap-retention has two conflicting schema identities; the other three have +> none.** ### The cheap next step is versioning, not unification @@ -153,7 +169,8 @@ So the minimal work that closes the real hole found above, in order: 1. **Every analysis-facing artifact gets its own mandatory schema identity** — one per family, e.g. `heap-retention`, `leak-growth`, `duplicate-immutable`, `propertychanged-storm` (names illustrative, not - normative). Today four of the five producers emit no version at all. + normative). Today three of the five producers emit no version at all, and + the two that do disagree with each other — see the family table above. 2. **Readers must validate family and major version.** Tolerant of unknown *fields*, intolerant of unknown *identity*: @@ -232,4 +249,9 @@ recommendation for a .NET auditor. - OwnAudit was read at a shallow clone of `main`; the collector inventory and schema-string findings are about that tree plus `Own.NET/audit/runtime/` at this branch's base commit. -- **Recorded, not scheduled.** No work item is filed off this note. +- **Recorded, not scheduled** — with one exception. The identity/validation + defect measured above is no longer an idea from a paper but a demonstrated + contract hole, so it has a written spec at + [`docs/tasks/runtime-artifact-identity.md`](../tasks/runtime-artifact-identity.md) + (status: *spec, ready to implement*, explicitly **not** inserted into the + current sequence). Everything else here remains recorded only. diff --git a/docs/tasks/runtime-artifact-identity.md b/docs/tasks/runtime-artifact-identity.md new file mode 100644 index 00000000..fffa3497 --- /dev/null +++ b/docs/tasks/runtime-artifact-identity.md @@ -0,0 +1,132 @@ +# Task — enforce runtime artifact family identity and major-version validation + +Status: **spec, ready to implement** — *not* scheduled into the current +sequence. See §8. + +Derived from [`docs/notes/leaf-dmir-assessment.md`](../notes/leaf-dmir-assessment.md), +where the defect was measured rather than proposed. This closes a contract hole +in the runtime arm. It deliberately does **not** open the shared-envelope +question, which that note argues must come *after* per-family identity, not +before. + +## 0. Goal + +Make every analysis-facing runtime artifact carry a mandatory family identity +and major version, and make every reader reject an artifact it cannot identify. +Do **not** change collection semantics, analysis semantics, or SARIF output. + +## 1. The defect (measured, not hypothetical) + +Five producer implementations emit four analysis-facing artifact families. All +four have an identity problem: + +| artifact family | producer(s) | identity today | +|---|---|---| +| heap-retention | `audit/runtime/RetentionPath` + `OwnAudit/src/OwnAudit.Runtime` | **two conflicting**: `own-runtime/1` vs `ownAudit/runtime/v1` | +| leak-growth | `audit/runtime/LeakHarness` | **none** | +| duplicate-immutable | `audit/runtime/DuplicateDetector` | **none** | +| propertychanged-storm | `audit/runtime/PropertyChangedStorm` | **none** | + +And nothing validates any of it. `OwnAudit/runtime/correlate.py` consumes only +`retained / type / count / expected / bytes / roots / kind / holder / member` +and never reads `schema`. **Any JSON containing a `retained` array is currently +accepted as the heap-retention contract.** The single assertion that exists +(`HeapCollectorContractTests.cs:83`) checks one producer against itself. + +The heap-retention split is a **lift-out artifact**, not stray junk: the +collector plan (2026-07-18) ports `stackpeek` into `OwnAudit.Runtime` while +`RetentionPath` remains canonical in Own.NET. Any fix must treat it as a +migration with a stated compatibility decision, not pick a winner silently. + +## 2. Scope + +**In scope** + +1. Assign **one schema identity per shipped analysis-facing artifact family** + (four families; names to be settled in review — not normative here). +2. Resolve the heap-retention identity conflict (`own-runtime/1` vs + `ownAudit/runtime/v1`) with an **explicit** compatibility/migration + decision covering both producers. +3. Make each reader reject: missing identity, wrong family, unsupported major. +4. Preserve forward compatibility: unknown **fields** under a known + family + major stay accepted. +5. Negative fixtures/tests for every rejection case (§3). +6. Preserve collection semantics unchanged: refusal/failure → exit 2 → **no + artifact**. + +**Out of scope (explicitly do not do)** + +- A common envelope across families. Deferred by design — commonality must be + proven by the four shipped schemas first. +- Any schema-field unification beyond identity + version. +- A `completeness` / `PARTIAL` enum of any kind. Coverage semantics belong to + the artifact family (see the note's three-way split), and heap-retention's + contract may well stay total-or-no-artifact. +- Changes to acquisition: no `.dmp`/`.etl` normalization, no new capture paths. +- New runtime analyses or new artifact families. +- Any change to SARIF semantics or to `ingest.py` output. + +## 3. Acceptance matrix (hard) + +Happy path alone does not close this defect. Required cases: + +| input | expected | +|---|---| +| correct family + supported major | accept | +| correct family + supported major + unknown extra fields | accept | +| missing `schema` | **reject** | +| `schema` of a different runtime family | **reject** | +| unknown major | **reject** | +| malformed required payload | **reject** | +| collector refusal | exit 2, no artifact — unchanged | + +The load-bearing test, which is what actually proves the hole is closed: + +> a **valid-looking `retained[]`** payload with a **wrong or missing `schema`** +> **MUST NOT** correlate. + +Without that case the change is a decorative `if schema != ...`. + +For the lift-out, one more: + +> a legacy-producer artifact and a new-producer artifact must each have +> **explicitly specified** behaviour — accepted, rejected, or accepted with a +> recorded deprecation. Not left to whichever string was typed first. + +## 4. Non-goals restated as a guard + +If a reviewer or implementer finds themselves designing a field that every +family would share, **stop** — that is the envelope work this task defers, and +taking it here converts a small correctness fix into an architecture change. + +## 5. Priority + +**Small correctness debt, high leverage.** Not a blocker, not urgent. + +- *Why file it:* the defect is real, tested cheaply, and its remediation cost + grows with every new artifact family. +- *Why not drop everything:* `correlate.py` behaves predictably today and no + false verdict from schema drift has been demonstrated. This is a latent + contract hole, not a fire. + +## 6. Repository boundary + +The change spans both repos: producers in `Own.NET/audit/runtime/` and +`OwnAudit/src/OwnAudit.Runtime`, readers in `OwnAudit/runtime/`. The +heap-retention decision (§2.2) is the coupling point and should be settled once, +in one place, before either side is edited. + +## 7. Risks / pitfalls + +- **Picking a canonical string silently** breaks the other producer. §3's + lift-out case exists to prevent exactly this. +- **Tolerant-reader drift**: rejecting unknown *fields* instead of unknown + *identity* would break forward compatibility, which is a deliberate and + correct property today. +- **Scope creep into the envelope** — see §4. + +## 8. Sequencing + +Record now, implement **after** the established boundary/gate work in flight. +This is explicitly not a blocker for the current PR sequence, and inserting it +there would turn a small fix into a detour. From 0e9de1958610a7a1619ffe2261d230df0d9d3519 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 16:56:38 +0000 Subject: [PATCH 5/6] docs(tasks): pin the no-issue-yet lifecycle in the spec itself The spec said 'not scheduled' but not what happens when it is, so the default next action was to copy 132 lines into a GitHub issue and mint a second normative surface -- the same duplicate-contract defect this task exists to fix, reproduced in our own process. Records the decision: this file stays canonical, no issue is filed at this stage, and when the task enters the execution queue the issue is a thin handle carrying title, a pointer to this spec, why-now, live dependencies and the PR link -- with scope and acceptance explicitly not duplicated. Also notes the second reason to wait: the task spans both repositories and its heap-retention compatibility decision is taken up front, so execution metadata frozen today would be stale by the time work starts. No code, no scope, no acceptance changes. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/tasks/runtime-artifact-identity.md | 40 +++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/docs/tasks/runtime-artifact-identity.md b/docs/tasks/runtime-artifact-identity.md index fffa3497..564331a5 100644 --- a/docs/tasks/runtime-artifact-identity.md +++ b/docs/tasks/runtime-artifact-identity.md @@ -130,3 +130,43 @@ in one place, before either side is edited. Record now, implement **after** the established boundary/gate work in flight. This is explicitly not a blocker for the current PR sequence, and inserting it there would turn a small fix into a detour. + +### This file is the canonical surface — deliberately no issue yet + +The lifecycle, decided when this spec was filed: + +```text +finding → docs/notes/leaf-dmir-assessment.md +demonstrated defect → THIS FILE (spec, ready to implement) + ── no GitHub issue at this stage ── +boundary/gate work done, task enters the execution queue + → thin GitHub issue (execution handle only) + → implementation PR + → this spec becomes the historical record +``` + +**No issue exists today, on purpose.** Filing one now would create a second +status surface for work we are deliberately not scheduling, and two copies of +scope/acceptance to drift apart — the exact defect this task exists to fix, +reproduced in our own process. + +There is a second reason to wait: §6 notes this spans both repositories and that +the heap-retention compatibility decision must be taken once, up front. Both +trees will move before this starts, so freezing execution metadata today buys +stale dependencies. + +**When it is scheduled**, the issue is a *handle*, not a specification — it must +not restate scope or acceptance: + +```text +Title: Enforce runtime artifact family identity and major-version validation + +Canonical spec: docs/tasks/runtime-artifact-identity.md +Why now: the sequenced boundary/gate work is complete; entering the queue. +Scope/acceptance: normative in the task spec. Do not duplicate here. +Depends/sequencing: +Implementation PR: TBD +``` + +That keeps GitHub's real advantages — assignability, labels/milestone, queue +position, PR linkage, closure — without minting a second normative document. From cd0d771ef73ae10bba2648d969b76274f1c5477b Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 17:39:50 +0000 Subject: [PATCH 6/6] =?UTF-8?q?docs:=20address=20review=20=E2=80=94=20add?= =?UTF-8?q?=20the=20Own.NET=20reader=20and=20scope=20the=20exit-2=20invari?= =?UTF-8?q?ant?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two valid findings from the Codex review on #328, both verified against the tree before acting. Reader scope was incomplete. audit/runtime/ingest.py is the active reader for leak-growth, duplicate-immutable and propertychanged-storm -- it takes --leak-harness / --duplicate-detector / --propertychanged-storm, does a bare json.loads and emits SARIF with no identity check -- but the spec listed readers only under OwnAudit/runtime/. Following it as written would have given three of four families an identity nothing was required to validate, which is the defect in miniature. Adds a reader table naming both repositories and points the negative cases at ingest.py's existing --selftest. The exit-2 invariant was over-generalized from RetentionPath's documented contract. Checked per producer: DuplicateDetector and PropertyChangedStorm catch Exception and return 2, but LeakHarness catches only ScenarioException, so a missing scenario file, a failed process launch, a missing window or a failed dump leaves Main with the unhandled-exception code. Requiring exit 2 across all four families would therefore change collection semantics, which the task's own out-of-scope list forbids. The requirement is now scoped to producers that already implement it, with normalizing LeakHarness called out as a separate labelled behaviour change. The same over-generalization is corrected in the assessment note's collection-outcome bullet. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/leaf-dmir-assessment.md | 9 ++++- docs/tasks/runtime-artifact-identity.md | 52 +++++++++++++++++++++---- 2 files changed, 52 insertions(+), 9 deletions(-) diff --git a/docs/notes/leaf-dmir-assessment.md b/docs/notes/leaf-dmir-assessment.md index eb08b80e..04054e2a 100644 --- a/docs/notes/leaf-dmir-assessment.md +++ b/docs/notes/leaf-dmir-assessment.md @@ -197,8 +197,13 @@ The caution raised earlier resolves cleanly once the word is broken up. These are three different states and are currently drifting toward one label: - **Collection outcome.** Success → an artifact may exist. Refusal or failure → - exit 2 and the artifact **must not** exist. *The existing fail-closed - invariant, unchanged.* + exit 2 and the artifact **must not** exist. *Unchanged where it already + holds* — and it does **not** hold everywhere: `RetentionPath` has the + documented contract, `DuplicateDetector` and `PropertyChangedStorm` catch + broadly and return 2, but `LeakHarness` catches only `ScenarioException`, so + a missing scenario file or a failed dump escapes `Main` with a different exit + code. Normalizing that is a behaviour change, not a clean-up; see the task + spec's caveat. - **Artifact validity.** Given an artifact: known schema, required fields, well-formed, producer contract satisfied → valid, else reject. *This is the layer that is missing today* (step 2 above). diff --git a/docs/tasks/runtime-artifact-identity.md b/docs/tasks/runtime-artifact-identity.md index 564331a5..49dbe17d 100644 --- a/docs/tasks/runtime-artifact-identity.md +++ b/docs/tasks/runtime-artifact-identity.md @@ -51,8 +51,10 @@ migration with a stated compatibility decision, not pick a winner silently. 4. Preserve forward compatibility: unknown **fields** under a known family + major stay accepted. 5. Negative fixtures/tests for every rejection case (§3). -6. Preserve collection semantics unchanged: refusal/failure → exit 2 → **no - artifact**. +6. Preserve collection semantics **as they are per producer** — do not + normalize them as a side effect of this task. See the caveat below: the + exit-2 rule is not currently universal, and making it so is a behaviour + change that this task does not carry. **Out of scope (explicitly do not do)** @@ -78,7 +80,25 @@ Happy path alone does not close this defect. Required cases: | `schema` of a different runtime family | **reject** | | unknown major | **reject** | | malformed required payload | **reject** | -| collector refusal | exit 2, no artifact — unchanged | +| collector refusal | whatever that producer does today — unchanged (see below) | + +### Caveat: "failure → exit 2" is *not* a universal invariant today + +Checked per producer rather than assumed from the documented `RetentionPath` +contract: + +| producer | failure behaviour | +|---|---| +| `RetentionPath` | documented 0/1/2 contract (`runtime-witness-operations.md`) | +| `DuplicateDetector` | `catch (Exception)` → `return 2` | +| `PropertyChangedStorm` | `catch (Exception)` → `return 2` | +| **`LeakHarness`** | catches **only `ScenarioException`** → 2. A missing scenario file, a failed process launch, a missing window or a failed dump throws out of `Main` and exits with the runtime's unhandled-exception code, **not** 2 | + +So requiring exit 2 across all four families would **change collection +semantics** — which §2's own out-of-scope list forbids. The invariant is +therefore scoped to producers that already implement it. Normalizing +`LeakHarness` failures is a **separate, explicitly-labelled behaviour change** +and is *not* part of this task. The load-bearing test, which is what actually proves the hole is closed: @@ -111,10 +131,28 @@ taking it here converts a small correctness fix into an architecture change. ## 6. Repository boundary -The change spans both repos: producers in `Own.NET/audit/runtime/` and -`OwnAudit/src/OwnAudit.Runtime`, readers in `OwnAudit/runtime/`. The -heap-retention decision (§2.2) is the coupling point and should be settled once, -in one place, before either side is edited. +The change spans both repos. + +**Producers:** `Own.NET/audit/runtime/` (`RetentionPath`, `LeakHarness`, +`DuplicateDetector`, `PropertyChangedStorm`) and `OwnAudit/src/OwnAudit.Runtime`. + +**Readers — both repositories, not just OwnAudit:** + +| reader | families it reads | +|---|---| +| `OwnAudit/runtime/correlate.py` | heap-retention | +| **`Own.NET/audit/runtime/ingest.py`** | **leak-growth, duplicate-immutable, propertychanged-storm** | + +`ingest.py` is the active reader for three of the four families — it takes +`--leak-harness` / `--duplicate-detector` / `--propertychanged-storm`, does a +bare `json.loads`, and converts to SARIF with no identity check whatsoever. +**Omitting it would be the defect in miniature**: three families would gain a +schema identity that nothing requires anyone to validate, leaving exactly the +hole this task exists to close. Its `--selftest` entry point is the natural home +for the negative cases in §3. + +The heap-retention decision (§2.2) is the coupling point and should be settled +once, in one place, before either side is edited. ## 7. Risks / pitfalls