From c94f419024886b55b4cc0ec69886d819e60fdefc Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 04:55:47 +0000 Subject: [PATCH 1/4] docs(notes): measured feasibility of LLVM-grade codegen for .NET, plus GC/LOH observability Two research notes from a design discussion, both recorded and neither scheduled, per the research-landscape-2026 discipline. llvm-codegen-feasibility.md answers "can .NET code get C/C++-grade compiler optimizations, and is a PoC doable?" with a committed, reproducible harness rather than an opinion. Three results decide it: - A naive CIL lowering (per-element bounds check with a noreturn throw, array length reloaded from the object header) switches LLVM's auto-vectorizer off entirely. Hoist the check and both blocked kernels vectorize. The enabling work is .NET-specific range-check elimination in the frontend, not anything LLVM does for free. - noalias bought 0% on every kernel measured, so the ownership-to-noalias-to- speed story that makes Rust fast does not reproduce here. This is the finding that says the idea is a codegen project, not an Own.NET project. - Hand-written C# Vector256 matched LLVM -O3 on one kernel and beat clang -O3 by 1.8x on another, so LLVM is automation rather than a capability ceiling. Also records what RyuJIT actually does (disassembly shows loop cloning, hoisted length/null checks, a bounds-check-free hot loop, strength reduction) and what it does not (vectorize, unroll) - and why no compiler hoists an opaque LINQ call out of a loop, which makes that a static-analysis target rather than a codegen one. gc-observability-and-loh.md takes the one correction worth having from the GCExperiment write-up: the LOH threshold is compared against full object size, so byte[84_999] lands on the LOH at 85,024 bytes and the "keep buffers under 85,000" folklore is off by a header. The detectability-matrix boundary is restated explicitly - cheaper GC observation is not an argument for static inference, and LOH fragmentation stays runtime-only. The harness enforces a 5000-call warmup and a cross-variant correctness gate because both traps were hit while producing these numbers: a short warmup inflated LLVM's advantage by ~40%, and an unequal accumulator width produced a flattering 33x that was simply less work. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/gc-observability-and-loh.md | 99 +++++++ .../llvm-codegen-feasibility-data/Kernels.cs | 168 +++++++++++ .../llvm-codegen-feasibility-data/Program.cs | 109 ++++++++ .../llvm-codegen-feasibility-data/README.md | 26 ++ .../bench.csproj | 10 + .../llvm-codegen-feasibility-data/checkfree.c | 13 + .../llvm-codegen-feasibility-data/lenprobe.c | 49 ++++ .../llvm-codegen-feasibility-data/matrix.c | 37 +++ .../llvm-codegen-feasibility-data/native.c | 29 ++ .../llvm-codegen-feasibility-data/native2.c | 31 +++ .../llvm-codegen-feasibility-data/run.sh | 46 +++ docs/notes/llvm-codegen-feasibility.md | 262 ++++++++++++++++++ 12 files changed, 879 insertions(+) create mode 100644 docs/notes/gc-observability-and-loh.md create mode 100644 docs/notes/llvm-codegen-feasibility-data/Kernels.cs create mode 100644 docs/notes/llvm-codegen-feasibility-data/Program.cs create mode 100644 docs/notes/llvm-codegen-feasibility-data/README.md create mode 100644 docs/notes/llvm-codegen-feasibility-data/bench.csproj create mode 100644 docs/notes/llvm-codegen-feasibility-data/checkfree.c create mode 100644 docs/notes/llvm-codegen-feasibility-data/lenprobe.c create mode 100644 docs/notes/llvm-codegen-feasibility-data/matrix.c create mode 100644 docs/notes/llvm-codegen-feasibility-data/native.c create mode 100644 docs/notes/llvm-codegen-feasibility-data/native2.c create mode 100755 docs/notes/llvm-codegen-feasibility-data/run.sh create mode 100644 docs/notes/llvm-codegen-feasibility.md diff --git a/docs/notes/gc-observability-and-loh.md b/docs/notes/gc-observability-and-loh.md new file mode 100644 index 00000000..cb96f53e --- /dev/null +++ b/docs/notes/gc-observability-and-loh.md @@ -0,0 +1,99 @@ +# GC observability and the LOH — what is worth adopting, and what stays runtime-only + +Working note. **Trigger:** the *GCExperiment* write-up ("Making .NET GC +behaviour observable"), proposed as material to fold into our runtime layer. +This note records what is genuinely new in it, corrects one number our own docs +imply, and pins the boundary that must not move. + +Companion to [`llvm-codegen-feasibility.md`](llvm-codegen-feasibility.md) (the +other half of the same discussion) and to +[P-034](../proposals/P-034-runtime-lifetime-guard.md), whose "ClrMD-free +complement" argument this reuses verbatim. + +## The one factual correction worth taking + +Our docs treat the Large Object Heap threshold as **85,000 bytes** of payload. +That is the documented constant, but it is not the predicate. The GC compares +the **full object size** — payload + object header + method table pointer + +alignment padding — so an array whose *payload* is comfortably under the limit +still lands on the LOH: + +> `byte[84_999]` → 85,023 bytes with the 24-byte array header → rounds to +> 85,024 → **allocated on the LOH.** + +The practical consequence, and the reason it is worth writing down: the familiar +"keep buffers under 85,000" folklore is **off by roughly one header**, and a +pool sized to exactly `85_000 - 1` is on the wrong heap. Anyone sizing a buffer +pool against the constant should size against `85_000 - 24 - padding`, or +simply measure. + +Where this touches our docs: [`ROADMAP.md`](../ROADMAP.md) and +[`Plan.md`](../../Plan.md) both list LOH fragmentation in the detectability +matrix. Neither states a threshold, so **neither is wrong** — but if a threshold +is ever quoted in a rule, a diagnostic message, or a talk, it must be the +full-object-size version, not the payload one. + +## What GCExperiment is, and what is actually adoptable + +Four self-contained experiments (LOH placement; generation promotion; +allocation pressure; LOH fragmentation) built on ordinary public APIs — +`GC.GetGeneration`, `GC.Collect` with forced modes, +`GC.WaitForPendingFinalizers`, `GCSettings.LargeObjectHeapCompactionMode`, +plus small `GCMonitor`/`GCInfo` helpers for snapshots and size estimation. It +also flags a real measurement trap: without `GC.KeepAlive`, the JIT can shorten +an object's lifetime and skew the result. + +The adoptable idea is **not** the GC content, which is well-trodden. It is the +**delivery shape**, and it is the same shape P-034 already argued for from a +different direction: + +> a lifetime/GC observation that runs in an ordinary `dotnet test`, on any OS, +> with no PerfView, no ETW, no Windows stand, and no ClrMD heap walk. + +Today our runtime layer ([`Plan.md`](../../Plan.md) §2, category 12) routes +*everything* GC-shaped to PerfView + ETW. That is correct for **evidence** and +badly overweight for **orientation** — it means no GC fact can be established in +CI, on Linux, or in a unit test. A `GC.GetGCMemoryInfo` / +`GC.CollectionCount(n)` snapshot around a scenario costs nothing and needs no +stand. + +So: a cheap in-test GC probe is a reasonable sibling to P-034's disposal +quarantine, under the same honest caveat P-034 already states — it proves *what +the counters did during this test*, bounded by test coverage, and it is blind to +*why*. It is a debug assertion, not an auditor. + +## The boundary that does not move + +**None of this makes LOH fragmentation statically detectable.** The matrix rows +stay exactly as written: + +- `ROADMAP.md`: LOH fragmentation → ❌ **impossible** (depends on runtime data + volume / GC timing) +- `Plan.md` category 12: heavy dictionaries / LOH fragmentation / Gen2 bloat → + **impossible** static → **RUNTIME** + +A GC probe is a **runtime witness**, and it lives on the runtime side of the +line the detectability matrix draws. The matrix exists specifically to stop +runtime-shaped problems from being hung on a static checker +([`Plan.md`](../../Plan.md) §1: it "*forbids*" exactly that, killing a class of +false positives in advance). Making GC behaviour cheaper to *observe* is not an +argument for making it *inferred*, and this note must not be cited as one. + +The one thing genuinely on the static side is unchanged and already ours: the +`ArrayPool`/`Span` misuse family (`POOL001`–`003`, P-007). Rent/return balance +is structurally visible; fragmentation is not. + +## Status + +**Recorded, not scheduled.** Per the `research-landscape-2026.md` discipline, +notes record and the ROADMAP schedules. Concretely, if anything is ever picked +up from here: + +1. **The LOH threshold correction** — free, and the only item with a + correctness argument behind it. Applies wherever a number gets quoted. +2. **A GC-counter probe as a P-034 sibling** — small, attaches to the open + question P-034 already asks (a new `Own.Diagnostics` package vs living in + OwnAudit's `runtime/`). Do not file it separately; it is the same decision. +3. **Nothing else.** The generation-promotion and allocation-pressure + experiments are educational rather than diagnostic, and we do not need to + re-derive published GC behaviour to ship a checker. diff --git a/docs/notes/llvm-codegen-feasibility-data/Kernels.cs b/docs/notes/llvm-codegen-feasibility-data/Kernels.cs new file mode 100644 index 00000000..71a93697 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/Kernels.cs @@ -0,0 +1,168 @@ +using System; +using System.Diagnostics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; + +// Kernels 2-4: is the LLVM advantage general, or confined to straight-line +// numeric loops? Run with a long warmup so every method is at tier-1. +static class Kernels +{ + const int N = 1 << 16; + const int Reps = 4000; + const int Warm = 5000; + + [MethodImpl(MethodImplOptions.NoInlining)] + static long SumManaged(int[] a) + { + long s = 0; + for (int i = 0; i < a.Length; i++) s += a[i]; + return s; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long FilterSumManaged(int[] a, int lo, int hi) + { + long s = 0; + for (int i = 0; i < a.Length; i++) + { + int v = a[i]; + if (v > lo && v < hi) s += v * 2; else s -= v; + } + return s; + } + + // K3 control A: hand-written branchless C# (encourage cmov, kill the + // misprediction cost without any SIMD). + [MethodImpl(MethodImplOptions.NoInlining)] + static long FilterSumBranchless(int[] a, int lo, int hi) + { + long s = 0; + for (int i = 0; i < a.Length; i++) + { + int v = a[i]; + int inRange = (v > lo & v < hi) ? 1 : 0; + s += inRange * (v * 2) + (1 - inRange) * (-v); + } + return s; + } + + // K3 control B: hand-written SIMD with a select -- the same transform + // LLVM's vectorizer applies automatically (if-conversion + vectorize). + [MethodImpl(MethodImplOptions.NoInlining)] + static long FilterSumSimd(int[] a, int lo, int hi) + { + int w = System.Runtime.Intrinsics.Vector256.Count, i = 0; + var vlo = System.Runtime.Intrinsics.Vector256.Create(lo); + var vhi = System.Runtime.Intrinsics.Vector256.Create(hi); + // Accumulate in 64-bit to match the scalar version's `long s` exactly: + // widen each 8x32 result to 2x 4x64 before adding. + var accLo = System.Runtime.Intrinsics.Vector256.Zero; + var accHi = System.Runtime.Intrinsics.Vector256.Zero; + for (; i <= a.Length - w; i += w) + { + var v = System.Runtime.Intrinsics.Vector256.LoadUnsafe(ref a[i]); + var mask = System.Runtime.Intrinsics.Vector256.GreaterThan(v, vlo) + & System.Runtime.Intrinsics.Vector256.LessThan(v, vhi); + var sel = System.Runtime.Intrinsics.Vector256.ConditionalSelect(mask, v + v, -v); + (var wLo, var wHi) = System.Runtime.Intrinsics.Vector256.Widen(sel); + accLo += wLo; accHi += wHi; + } + long s = System.Runtime.Intrinsics.Vector256.Sum(accLo) + + System.Runtime.Intrinsics.Vector256.Sum(accHi); + for (; i < a.Length; i++) + { + int v = a[i]; + if (v > lo && v < hi) s += v * 2; else s -= v; + } + return s; + } + + sealed class Node { public Node Next; public long Value; } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long ChaseManaged(Node head, long n) + { + long s = 0; + for (long i = 0; i < n && head != null; i++) { s += head.Value; head = head.Next; } + return s; + } + + [DllImport("native2", EntryPoint = "sum_native")] + static extern unsafe long SumNative(int* a, long n, long la); + [DllImport("native2", EntryPoint = "filter_sum_native")] + static extern unsafe long FilterSumNative(int* a, long n, long la, int lo, int hi); + // check-free variants: the frontend hoisted the range check out of the loop + [DllImport("native3", EntryPoint = "sum_free")] + static extern unsafe long SumFree(int* a, long n, long la); + [DllImport("native3", EntryPoint = "filter_sum_free")] + static extern unsafe long FilterSumFree(int* a, long n, long la, int lo, int hi); + + static double Time(Action f, int reps) + { + var sw = Stopwatch.StartNew(); + for (int r = 0; r < reps; r++) f(); + sw.Stop(); + return sw.Elapsed.TotalMilliseconds; + } + + public static unsafe void Run() + { + var a = new int[N]; + var rnd = new Random(42); + for (int i = 0; i < N; i++) a[i] = rnd.Next(0, 1000); + + long sink = 0; + for (int r = 0; r < Warm; r++) sink += SumManaged(a) + FilterSumManaged(a, 200, 800) + + FilterSumBranchless(a, 200, 800) + FilterSumSimd(a, 200, 800); + fixed (int* pa = a) + { + for (int r = 0; r < Warm; r++) sink += SumNative(pa, N, N) + FilterSumNative(pa, N, N, 200, 800) + + SumFree(pa, N, N) + FilterSumFree(pa, N, N, 200, 800); + + int* p = pa; + double t; + double elems = (double)N * Reps; + + Console.WriteLine($"\n-- K2 reduction (sum) --"); + t = Time(() => sink += SumManaged(a), Reps); + double k2m = elems / t / 1e6; + Console.WriteLine($"RyuJIT managed : {t,8:F1} ms {k2m,6:F2} Gelem/s 1.00x"); + t = Time(() => sink += SumNative(p, N, N), Reps); + Console.WriteLine($"LLVM -O3 w/ check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k2m,5:F2}x"); + t = Time(() => sink += SumFree(p, N, N), Reps); + Console.WriteLine($"LLVM -O3 no check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k2m,5:F2}x"); + + // correctness gate: every K3 variant must return the SAME value. + long r0 = FilterSumManaged(a, 200, 800), r1 = FilterSumBranchless(a, 200, 800), + r2 = FilterSumSimd(a, 200, 800), r3 = FilterSumNative(p, N, N, 200, 800); + long r4 = FilterSumFree(p, N, N, 200, 800); + if (r0 != r1 || r0 != r2 || r0 != r3 || r0 != r4) + throw new Exception($"K3 MISMATCH scalar={r0} branchless={r1} simd={r2} native={r3} free={r4}"); + long q0 = SumManaged(a), q1 = SumNative(p, N, N), q2 = SumFree(p, N, N); + if (q0 != q1 || q0 != q2) throw new Exception($"K2 MISMATCH {q0} vs {q1} vs {q2}"); + Console.WriteLine($"\n-- K3 data-dependent branch (filter+sum) -- [all variants agree: {r0}]"); + t = Time(() => sink += FilterSumManaged(a, 200, 800), Reps); + double k3m = elems / t / 1e6; + Console.WriteLine($"RyuJIT managed : {t,8:F1} ms {k3m,6:F2} Gelem/s 1.00x"); + t = Time(() => sink += FilterSumBranchless(a, 200, 800), Reps); + Console.WriteLine($"RyuJIT branchless: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k3m,5:F2}x"); + t = Time(() => sink += FilterSumSimd(a, 200, 800), Reps); + Console.WriteLine($"RyuJIT SIMD+sel : {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k3m,5:F2}x"); + t = Time(() => sink += FilterSumNative(p, N, N, 200, 800), Reps); + Console.WriteLine($"LLVM -O3 w/ check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k3m,5:F2}x"); + t = Time(() => sink += FilterSumFree(p, N, N, 200, 800), Reps); + Console.WriteLine($"LLVM -O3 no check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k3m,5:F2}x"); + } + + // K4: managed pointer chasing has no native counterpart worth timing + // (the object graph lives in the GC heap); measured to show the shape + // of code where no vectorizer of any kind helps. + const int Len = 1 << 14; + Node head = null; + for (int i = 0; i < Len; i++) head = new Node { Next = head, Value = i }; + double tc = Time(() => sink += ChaseManaged(head, Len), Reps / 4); + Console.WriteLine($"\n-- K4 pointer chase (managed only) --"); + Console.WriteLine($"RyuJIT managed : {tc,8:F1} ms {(double)Len * (Reps / 4) / tc / 1e6,6:F2} Gnode/s"); + Console.WriteLine($"\n(sink={sink})"); + } +} diff --git a/docs/notes/llvm-codegen-feasibility-data/Program.cs b/docs/notes/llvm-codegen-feasibility-data/Program.cs new file mode 100644 index 00000000..45826e38 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/Program.cs @@ -0,0 +1,109 @@ +using System; +using System.Diagnostics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +static class Bench +{ + const int N = 1 << 16; // 64K ints per array = 256 KB, fits in L2 + const int Reps = 4000; + + [MethodImpl(MethodImplOptions.NoInlining)] + static void AxpyManaged(int[] dst, int[] a, int[] b) + { + for (int i = 0; i < dst.Length; i++) + dst[i] = a[i] + b[i] * 3; + } + + // Control: what a C# developer can already write today, no LLVM involved. + [MethodImpl(MethodImplOptions.NoInlining)] + static void AxpySimd(int[] dst, int[] a, int[] b) + { + int w = Vector256.Count, i = 0; + var three = Vector256.Create(3); + for (; i <= dst.Length - w; i += w) + { + var va = Vector256.LoadUnsafe(ref a[i]); + var vb = Vector256.LoadUnsafe(ref b[i]); + (va + vb * three).StoreUnsafe(ref dst[i]); + } + for (; i < dst.Length; i++) dst[i] = a[i] + b[i] * 3; + } + + // Control 2: hand-SIMD + 4x unroll, i.e. exactly what LLVM's vectorizer + // chose on its own (width 8, interleave 4). Written by hand in C#. + [MethodImpl(MethodImplOptions.NoInlining)] + static void AxpySimdUnrolled(int[] dst, int[] a, int[] b) + { + int w = Vector256.Count, i = 0; + var three = Vector256.Create(3); + for (; i <= dst.Length - 4 * w; i += 4 * w) + { + (Vector256.LoadUnsafe(ref a[i]) + Vector256.LoadUnsafe(ref b[i]) * three).StoreUnsafe(ref dst[i]); + (Vector256.LoadUnsafe(ref a[i + w]) + Vector256.LoadUnsafe(ref b[i + w]) * three).StoreUnsafe(ref dst[i + w]); + (Vector256.LoadUnsafe(ref a[i + 2 * w]) + Vector256.LoadUnsafe(ref b[i + 2 * w]) * three).StoreUnsafe(ref dst[i + 2 * w]); + (Vector256.LoadUnsafe(ref a[i + 3 * w]) + Vector256.LoadUnsafe(ref b[i + 3 * w]) * three).StoreUnsafe(ref dst[i + 3 * w]); + } + for (; i < dst.Length; i++) dst[i] = a[i] + b[i] * 3; + } + + [DllImport("native", EntryPoint = "axpy_noalias")] + static extern unsafe void AxpyNoalias(int* dst, int* a, int* b, long n, long ld, long la, long lb); + + [DllImport("native", EntryPoint = "axpy_alias")] + static extern unsafe void AxpyAlias(int* dst, int* a, int* b, long n, long ld, long la, long lb); + + static unsafe void Main() + { + var dst = new int[N]; var a = new int[N]; var b = new int[N]; + for (int i = 0; i < N; i++) { a[i] = i; b[i] = N - i; } + + // Warm up well past the tier-1 / OSR thresholds. 200 reps is NOT + // enough -- it leaves tier-0 code in the measurement (see the note). + for (int r = 0; r < 5000; r++) AxpyManaged(dst, a, b); + for (int r = 0; r < 5000; r++) AxpySimd(dst, a, b); + for (int r = 0; r < 5000; r++) AxpySimdUnrolled(dst, a, b); + fixed (int* pd = dst, pa = a, pb = b) + { + for (int r = 0; r < 5000; r++) AxpyNoalias(pd, pa, pb, N, N, N, N); + for (int r = 0; r < 5000; r++) AxpyAlias(pd, pa, pb, N, N, N, N); + } + + long checksum = 0; + var sw = new Stopwatch(); + + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpyManaged(dst, a, b); + sw.Stop(); var tManaged = sw.Elapsed.TotalMilliseconds; checksum += dst[N - 1]; + + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpySimd(dst, a, b); + sw.Stop(); var tSimd = sw.Elapsed.TotalMilliseconds; checksum += dst[N - 1]; + + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpySimdUnrolled(dst, a, b); + sw.Stop(); var tSimdU = sw.Elapsed.TotalMilliseconds; checksum += dst[N - 1]; + + fixed (int* pd = dst, pa = a, pb = b) + { + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpyNoalias(pd, pa, pb, N, N, N, N); + sw.Stop(); var tNoalias = sw.Elapsed.TotalMilliseconds; checksum += dst[N - 1]; + + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpyAlias(pd, pa, pb, N, N, N, N); + sw.Stop(); var tAlias = sw.Elapsed.TotalMilliseconds; + + double elems = (double)N * Reps; + Console.WriteLine($"N={N} reps={Reps} checksum={checksum}"); + Console.WriteLine($"RyuJIT managed : {tManaged,8:F1} ms {elems / tManaged / 1e6,6:F2} Gelem/s 1.00x"); + Console.WriteLine($"RyuJIT Vector256 : {tSimd,8:F1} ms {elems / tSimd / 1e6,6:F2} Gelem/s {tManaged / tSimd,5:F2}x"); + Console.WriteLine($"RyuJIT Vec256 x4 : {tSimdU,8:F1} ms {elems / tSimdU / 1e6,6:F2} Gelem/s {tManaged / tSimdU,5:F2}x"); + Console.WriteLine($"LLVM -O3 noalias : {tNoalias,8:F1} ms {elems / tNoalias / 1e6,6:F2} Gelem/s {tManaged / tNoalias,5:F2}x"); + Console.WriteLine($"LLVM -O3 alias : {tAlias,8:F1} ms {elems / tAlias / 1e6,6:F2} Gelem/s {tManaged / tAlias,5:F2}x"); + } + + Kernels.Run(); + } +} diff --git a/docs/notes/llvm-codegen-feasibility-data/README.md b/docs/notes/llvm-codegen-feasibility-data/README.md new file mode 100644 index 00000000..d914e0f5 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/README.md @@ -0,0 +1,26 @@ +# Harness for `../llvm-codegen-feasibility.md` + +Reproduces every number in the note. Requires `clang` (tested 18.1.3), the +.NET SDK 9 (tested 9.0.316), and an x86-64-v3 CPU. + +```console +$ ./run.sh [workdir] +``` + +| File | Role | +|---|---| +| `lenprobe.c` | 2x2: array length re-loaded vs hoisted, x aliasing unknown vs `restrict` | +| `matrix.c` | 2x2: aliasing x range-check-eliminated, lengths passed as arguments | +| `checkfree.c` | the K2/K3 kernels with the range check hoisted out of the loop | +| `native.c` | K1 (`dst[i] = a[i] + b[i]*3`), noalias and aliasing variants | +| `native2.c` | K2 reduction, K3 data-dependent branch, K4 pointer chase | +| `Program.cs` | K1 harness: RyuJIT scalar / `Vector256` / `Vector256` x4 / native | +| `Kernels.cs` | K2-K4 harness, incl. the correctness gate across all K3 variants | + +Two things the harness enforces on purpose, both because they were got wrong +first (see "Two measurement traps" in the note): + +- **`DOTNET_TieredCompilation=0` plus a 5000-call warmup.** A short warmup + measures partly-tier-0 code and inflates LLVM's advantage by ~40%. +- **A correctness gate.** All K3 variants must return `25830282`; an unequal + accumulator width silently makes one variant do less work. diff --git a/docs/notes/llvm-codegen-feasibility-data/bench.csproj b/docs/notes/llvm-codegen-feasibility-data/bench.csproj new file mode 100644 index 00000000..c0f7e09f --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/bench.csproj @@ -0,0 +1,10 @@ + + + Exe + net9.0 + disable + true + true + false + + diff --git a/docs/notes/llvm-codegen-feasibility-data/checkfree.c b/docs/notes/llvm-codegen-feasibility-data/checkfree.c new file mode 100644 index 00000000..f0a6fe92 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/checkfree.c @@ -0,0 +1,13 @@ +#include +__attribute__((noreturn)) static void own_throw_ioor(void) { abort(); } +// same kernels, but the range check is hoisted out of the loop (frontend RCE) +long sum_free(const int *restrict a, long n, long la) { + if (n > la) own_throw_ioor(); + long s = 0; for (long i = 0; i < n; i++) s += a[i]; return s; +} +long filter_sum_free(const int *restrict a, long n, long la, int lo, int hi) { + if (n > la) own_throw_ioor(); + long s = 0; + for (long i = 0; i < n; i++) { int v = a[i]; if (v > lo && v < hi) s += v*2; else s -= v; } + return s; +} diff --git a/docs/notes/llvm-codegen-feasibility-data/lenprobe.c b/docs/notes/llvm-codegen-feasibility-data/lenprobe.c new file mode 100644 index 00000000..470ae98c --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/lenprobe.c @@ -0,0 +1,49 @@ +// Isolate the real blocker: is it aliasing, the range check, or the fact that +// `ldlen` is a MEMORY LOAD inside the loop (array length not known immutable)? +#include +void own_throw_ioor(void) __attribute__((noreturn)); +typedef struct { long len; int data[]; } Arr; // a .NET array object + +// 1. length reloaded from the object header every iteration, no alias info. +// This is what a naive CIL->LLVM frontend emits for ldlen. +void p1_reload_alias(Arr *dst, Arr *a, Arr *b, long n) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)a->len) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)b->len) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)dst->len) own_throw_ioor(); + dst->data[i] = a->data[i] + b->data[i] * 3; + } +} + +// 2. same, but the three objects are known distinct (type-safety / ownership). +void p2_reload_noalias(Arr *restrict dst, Arr *restrict a, Arr *restrict b, long n) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)a->len) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)b->len) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)dst->len) own_throw_ioor(); + dst->data[i] = a->data[i] + b->data[i] * 3; + } +} + +// 3. lengths hoisted once (frontend knows array length is IMMUTABLE), aliasing +// still unknown, per-element range check kept. +void p3_hoistlen_alias(Arr *dst, Arr *a, Arr *b, long n) { + long la = a->len, lb = b->len, ld = dst->len; + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst->data[i] = a->data[i] + b->data[i] * 3; + } +} + +// 4. lengths hoisted + noalias. +void p4_hoistlen_noalias(Arr *restrict dst, Arr *restrict a, Arr *restrict b, long n) { + long la = a->len, lb = b->len, ld = dst->len; + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst->data[i] = a->data[i] + b->data[i] * 3; + } +} diff --git a/docs/notes/llvm-codegen-feasibility-data/matrix.c b/docs/notes/llvm-codegen-feasibility-data/matrix.c new file mode 100644 index 00000000..13825850 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/matrix.c @@ -0,0 +1,37 @@ +// 2x2: {alias info} x {range checks eliminated} -> does LLVM -O3 vectorize? +#include +void own_throw_ioor(void) __attribute__((noreturn)); + +// --- A: aliasing unknown, per-element range check (naive CIL lowering) --- +void a_alias_check(int *dst, const int *a, const int *b, long n, long ld, long la, long lb) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst[i] = a[i] + b[i] * 3; + } +} + +// --- B: noalias, per-element range check --- +void b_noalias_check(int *restrict dst, const int *restrict a, const int *restrict b, + long n, long ld, long la, long lb) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst[i] = a[i] + b[i] * 3; + } +} + +// --- C: aliasing unknown, checks hoisted out of the loop (frontend RCE) --- +void c_alias_nocheck(int *dst, const int *a, const int *b, long n, long ld, long la, long lb) { + if (n > la || n > lb || n > ld) own_throw_ioor(); + for (long i = 0; i < n; i++) dst[i] = a[i] + b[i] * 3; +} + +// --- D: noalias + checks hoisted --- +void d_noalias_nocheck(int *restrict dst, const int *restrict a, const int *restrict b, + long n, long ld, long la, long lb) { + if (n > la || n > lb || n > ld) own_throw_ioor(); + for (long i = 0; i < n; i++) dst[i] = a[i] + b[i] * 3; +} diff --git a/docs/notes/llvm-codegen-feasibility-data/native.c b/docs/notes/llvm-codegen-feasibility-data/native.c new file mode 100644 index 00000000..c75a4e2e --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/native.c @@ -0,0 +1,29 @@ +// Native side of the RyuJIT-vs-LLVM comparison. Same kernel, same semantics +// (per-element range check that traps), compiled by clang -O3. +#include +#include + +__attribute__((noreturn)) static void own_throw_ioor(void) { abort(); } + +// noalias + lengths hoisted (the "frontend knows array length is immutable" +// variant that LLVM vectorizes). +void axpy_noalias(int *restrict dst, const int *restrict a, const int *restrict b, + long n, long ld, long la, long lb) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst[i] = a[i] + b[i] * 3; + } +} + +// same, but aliasing unknown -> LLVM emits a runtime alias check + versioned loop. +void axpy_alias(int *dst, const int *a, const int *b, + long n, long ld, long la, long lb) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst[i] = a[i] + b[i] * 3; + } +} diff --git a/docs/notes/llvm-codegen-feasibility-data/native2.c b/docs/notes/llvm-codegen-feasibility-data/native2.c new file mode 100644 index 00000000..7315d91d --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/native2.c @@ -0,0 +1,31 @@ +#include +__attribute__((noreturn)) static void own_throw_ioor(void) { abort(); } + +// K2: reduction. +long sum_native(const int *restrict a, long n, long la) { + long s = 0; + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + s += a[i]; + } + return s; +} + +// K3: data-dependent branch -- shaped like business logic, not a math kernel. +long filter_sum_native(const int *restrict a, long n, long la, int lo, int hi) { + long s = 0; + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + int v = a[i]; + if (v > lo && v < hi) s += v * 2; else s -= v; + } + return s; +} + +// K4: pointer-chasing / indirection -- the shape of object-graph traversal. +typedef struct Node { struct Node *next; long value; } Node; +long chase_native(Node *head, long n) { + long s = 0; + for (long i = 0; i < n && head; i++) { s += head->value; head = head->next; } + return s; +} diff --git a/docs/notes/llvm-codegen-feasibility-data/run.sh b/docs/notes/llvm-codegen-feasibility-data/run.sh new file mode 100755 index 00000000..7d90e32a --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/run.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# Reproduce the measurements in ../llvm-codegen-feasibility.md. +# +# Requires: clang (tested: 18.1.3), .NET SDK 9 (tested: 9.0.316), x86-64-v3 CPU. +# Usage: ./run.sh [workdir] +set -euo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +WORK="${1:-$(mktemp -d)}" +mkdir -p "$WORK/cs" + +echo "== part 1: which loop shapes does LLVM -O3 vectorize? ==" +for f in lenprobe matrix; do + echo "--- $f.c ---" + clang -O3 -march=x86-64-v3 -c "$HERE/$f.c" -o /dev/null \ + -Rpass=loop-vectorize -Rpass-missed=loop-vectorize 2>&1 | grep remark || true +done +echo "--- checkfree.c (range check hoisted out of the loop) ---" +clang -O3 -march=x86-64-v3 -c "$HERE/checkfree.c" -o /dev/null \ + -Rpass=loop-vectorize -Rpass-missed=loop-vectorize 2>&1 | grep remark || true + +echo +echo "== part 2: RyuJIT vs LLVM -O3, same kernels ==" +cp "$HERE"/{Program.cs,Kernels.cs,bench.csproj} "$WORK/cs/" +clang -O3 -march=x86-64-v3 -shared -fPIC "$HERE/native.c" -o "$WORK/cs/libnative.so" +clang -O3 -march=x86-64-v3 -shared -fPIC "$HERE/native2.c" -o "$WORK/cs/libnative2.so" +# native3 = native2 kernels + the check-hoisted variants from checkfree.c +cat "$HERE/native2.c" > "$WORK/native3.c" +sed '/^#include/d; /own_throw_ioor(void) { abort/d' "$HERE/checkfree.c" >> "$WORK/native3.c" +clang -O3 -march=x86-64-v3 -shared -fPIC "$WORK/native3.c" -o "$WORK/cs/libnative3.so" + +cd "$WORK/cs" +dotnet build -c Release -v q --nologo +cp ./*.so bin/Release/net9.0/ + +# TieredCompilation=0 measures steady-state code. With tiering on, the default +# warmup is NOT enough and the managed numbers come out ~1.4x pessimistic -- +# see the "Two measurement traps" section of the note. +for run in 1 2 3; do + DOTNET_TieredCompilation=0 dotnet bin/Release/net9.0/bench.dll + echo +done + +echo "== part 3: RyuJIT disassembly of the scalar loop ==" +DOTNET_JitDisasm="AxpyManaged" DOTNET_TieredCompilation=0 \ + dotnet bin/Release/net9.0/bench.dll 2>&1 | sed -n '/Assembly listing/,/^$/p' | head -60 diff --git a/docs/notes/llvm-codegen-feasibility.md b/docs/notes/llvm-codegen-feasibility.md new file mode 100644 index 00000000..1a717b32 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility.md @@ -0,0 +1,262 @@ +# Can .NET code get C/C++-grade compiler optimizations? — a measured answer + +Working note. **Trigger:** a design discussion asked whether .NET code could be +pushed through some representation where LLVM's "insane" C/C++ optimizations +apply, and whether a proof-of-concept is doable on a shoestring. A second +question rode along: *what does .NET actually optimize in our code?* — prompted +by an expensive LINQ query that visibly re-ran every iteration of a `for` loop +on .NET Framework 4.7.2. + +This note answers both **with measurements, not opinion**. The harness is +committed in [`llvm-codegen-feasibility-data/`](llvm-codegen-feasibility-data/) +and reproduced by `run.sh`, so every number below is falsifiable. + +**Bottom line up front:** + +1. **PoC is doable** — the pipeline is alive and the measurement ladder below + already runs. But it is a *codegen* project, not an Own.NET project. +2. **The headline result is negative for us.** The "ownership → `noalias` → + speed" story that makes Rust fast **did not reproduce**: alias metadata + bought **0%** on every kernel measured. The thing that unlocks LLVM is + **range-check elimination in the frontend**, which is ordinary compiler work + with no ownership content. +3. **LLVM is not a capability ceiling C# cannot reach.** Hand-written C# + `Vector256` code matched LLVM `-O3` on one kernel and **beat clang `-O3` by + 1.8×** on another. What LLVM sells is *automation*, not headroom. +4. RyuJIT is **already better than the folklore** — it hoists, clones loops, and + eliminates bounds checks. What it does not do is **vectorize or unroll**. + +## Method + +- **Environment.** Intel Xeon @ 2.10GHz, 4 vCPU, AVX-512 (`avx512f/bw/dq/vl/cd/ + ifma/vbmi`), Linux 6.18.5. clang/LLVM **18.1.3**, .NET SDK **9.0.316** + (runtime 9.0.18), `linux-x64`. Native side always + `clang -O3 -march=x86-64-v3`. +- **Shape of the comparison.** The same kernel is written twice — once in C# + (run by RyuJIT) and once in C (compiled by clang `-O3`, called via + `DllImport` over `fixed` pointers). The C is deliberately written to mimic + *naive CIL lowering*: an explicit per-element range check that calls a + `noreturn` throw helper, and (where noted) the array length re-loaded from + the object header rather than hoisted. +- **Controls are the point.** Every kernel carries a hand-written C# SIMD + variant, because "LLVM beats scalar C#" is a boring claim; "LLVM beats + *well-written* C#" is the claim that would justify building anything. +- **Correctness gate.** All four K3 variants (scalar, branchless, hand-SIMD, + native) must return the identical value or the harness throws. They agree on + `25830282`. An early version of the SIMD control accumulated in 32-bit while + the scalar accumulated in 64-bit — it produced a flattering **33×** that was + simply *less work*. The gate exists because that mistake was made here. +- **Steady state.** Reported numbers use `DOTNET_TieredCompilation=0` plus a + 5000-call warmup. See the measurement traps below — this is not a detail. + +### Two measurement traps (both hit during this work) + +- **Warmup.** With tiering on and a 200-call warmup, the managed kernels + measured **1.4× slower** than steady state — and that inflated LLVM's apparent + advantage from **3.3× to 4.6×**. The first draft of this note would have + overstated the result by 40%. 4000 *measured* iterations did not save it; + only a longer warmup or `TieredCompilation=0` did. +- **Unequal work.** See the correctness gate above. + +## Result 1 — what actually blocks LLVM on CIL-shaped code + +`lenprobe.c`, 2×2 over the same `dst[i] = a[i] + b[i]*3` loop +(`-Rpass=loop-vectorize`): + +| | length re-loaded each iteration (naive `ldlen`) | length hoisted once (immutability known) | +|---|---|---| +| **aliasing unknown** | ❌ not vectorized | ✅ vectorized (width 8, interleave 2) | +| **`noalias` (`restrict`)** | ❌ not vectorized | ✅ vectorized (width 8, interleave 2) | + +And `checkfree.c` vs `native2.c`, over the reduction and branchy kernels: + +| loop body | vectorized? | +|---|---| +| per-element range check + `noreturn` throw | ❌ **not vectorized** | +| range check hoisted out of the loop | ✅ vectorized (width 4, interleave 4) | + +**Reading.** A naive CIL→LLVM frontend emits, for every single `ldelem`, a +bounds check whose failure edge calls a `noreturn` throw helper. That extra loop +exit **switches LLVM's vectorizer off entirely** — and re-loading the array +length from the object header does the same, independently. `noalias` changes +**nothing** in either row. + +So the load-bearing contribution a CIL→LLVM frontend must make is *.NET-specific +invariants* — array-length immutability, and range-check elimination — **before** +LLVM ever sees the IR. LLVM will not recover them for you. This inverts the +intuition the idea started from: the win is not "hand CIL to LLVM and collect +C-grade optimizations", it is "do the .NET-specific proof work yourself, and +LLVM's vectorizer is the reward." + +## Result 2 — RyuJIT vs LLVM, measured + +Median of 3 runs, steady state. `Gelem/s` = elements processed per second. + +**K1 — `dst[i] = a[i] + b[i]*3` (straight-line numeric):** + +| variant | Gelem/s | ×scalar | +|---|---|---| +| RyuJIT scalar | 1.85 | 1.00× | +| RyuJIT `Vector256`, no unroll | 5.4 | 2.9× | +| **RyuJIT `Vector256` ×4 unrolled** | **6.4** | **3.5×** | +| LLVM `-O3`, `noalias` | 6.1 | 3.3× | +| LLVM `-O3`, aliasing unknown | 6.1 | 3.3× | + +**K2 — reduction (`s += a[i]`):** + +| variant | Gelem/s | ×scalar | +|---|---|---| +| RyuJIT scalar | 2.75 | 1.00× | +| LLVM `-O3`, per-element check | 2.75 | **1.00×** | +| LLVM `-O3`, check hoisted | 11.1 | 4.0× | + +**K3 — data-dependent branch (`if (v > lo && v < hi) s += v*2; else s -= v`), +random data, ~unpredictable branch — the shape of business logic:** + +| variant | Gelem/s | ×scalar | +|---|---|---| +| RyuJIT scalar | 0.22 | 1.00× | +| RyuJIT "branchless" (multiply trick) | 0.18 | **0.80×** | +| **RyuJIT hand-SIMD + `ConditionalSelect`** | **5.0** | **22.5×** | +| LLVM `-O3`, per-element check | 1.39 | 6.2× | +| LLVM `-O3`, check hoisted | 2.81 | 12.5× | + +**K4 — pointer chase over a managed object graph:** 0.55 Gnode/s. No vectorizer +of any kind applies; this is latency-bound and included to mark the boundary. + +### Reading the table + +- **`noalias` bought nothing.** K1: 6.1 vs 6.1. Over a 64K-element loop, LLVM's + runtime alias check amortizes to zero and the versioned loop runs at the same + speed as the `restrict` one. **The ownership→speed story does not reproduce + here.** This is the single most decision-relevant number in the note, and it + is the one that says *this is not an Own.NET project*. +- **LLVM did not vectorize K2/K3 at all** — inspection of the assembly shows 0 + `ymm`/`zmm` registers in `filter_sum_native` and 2 `cmov`s. Its entire 6.2× on + K3 is **if-conversion** (branch → `cmov`, killing mispredictions), not SIMD. + The bounds check blocked the vectorizer, exactly as Result 1 predicts. +- **Hand-written C# wins where it is written.** C# `Vector256` ×4 (6.4) edges + out LLVM on K1 (6.1), and hand-SIMD C# on K3 (5.0) **beats clang `-O3` (2.8) + by 1.8×** — because a human applied if-conversion *and* vectorization where + LLVM managed only the former. +- **Micro-optimizing by hand can backfire.** The "branchless" multiply trick + made K3 **20% slower** than the naive branch. RyuJIT did not turn it into a + `cmov`; it just did more arithmetic. + +## Result 3 — what .NET actually optimizes (the LINQ-in-a-loop question) + +RyuJIT disassembly of the scalar K1 loop (`DOTNET_JitDisasm`, FullOpts) — +abridged: + +```asm +G_M000_IG02: mov ecx, dword ptr [rdi+0x08] ; dst.Length -- hoisted, loaded ONCE +G_M000_IG03: test rsi, rsi ; null checks -- hoisted + cmp dword ptr [rsi+0x08], ecx ; a.Length >= dst.Length -- hoisted + cmp dword ptr [rdx+0x08], ecx ; b.Length >= dst.Length -- hoisted +G_M000_IG04: mov r9d, dword ptr [rsi+4*r8+0x10] + mov r10d, dword ptr [rdx+4*r8+0x10] + lea r10d, [r10+2*r10] ; *3 -> lea (strength reduction) + add r9d, r10d + mov dword ptr [rdi+4*r8+0x10], r9d + inc eax + cmp ecx, eax + jg SHORT G_M000_IG04 ; hot loop: ZERO bounds checks +``` + +That is **loop cloning**: RyuJIT proved the safe precondition once, then emits a +fast loop with **no bounds checks at all**, keeping a checked clone (`IG06`) as +fallback. Plus hoisting of lengths and null checks, and strength reduction +(`*3` → `lea`). RyuJIT is doing real optimization work. + +**What it does not do: vectorize or unroll.** That is the whole 3.3× gap on K1, +and auto-vectorization remains an open request upstream +([dotnet/runtime#11263](https://github.com/dotnet/runtime/issues/11263)). + +**So why was the LINQ query not hoisted out of that 4.7.2 loop?** Not a JIT bug, +and LLVM would not have fixed it either. A LINQ chain is a sequence of +**opaque, allocating, interface-dispatched calls**. Loop-invariant code motion +may only hoist an expression it can prove side-effect-free, and nothing in the +CLR gives the JIT purity information about `Where`/`Select`/`Any` over an +arbitrary user predicate. No compiler in this note's data — RyuJIT *or* clang +`-O3` — hoists an opaque call out of a loop; it is not a legal transform without +a purity proof. On 4.7.2 it is worse still: no dynamic PGO, no guarded +devirtualization, so the delegate and enumerator calls stay indirect. + +The actionable consequence is the interesting one: **this class of bug is a +static-analysis target, not a codegen target.** "Loop-invariant expensive query +evaluated inside a loop" is structurally visible in source — which is Own.NET's +existing business — whereas no amount of LLVM would touch it. If any part of +this discussion deserves to become a rule, it is that one, and it has nothing to +do with LLVM. + +## Landscape — who has already tried this + +| Effort | What it is | Status | +|---|---|---| +| **Unity Burst** | LLVM over a C# subset (HPC#), the real proof this works — auto-vectorization, `[NoAlias]`, 10–100× on compute kernels | **Alive, shipping.** Buys its wins by *restricting the language* (no GC, no classes, no exceptions in kernels) | +| **NativeAOT-LLVM** | `dotnet/runtimelab`, LLVM backend for NativeAOT, primarily WebAssembly | Experimental branch; LLVM also used as object writer in shipping NativeAOT | +| **LLILC** | Microsoft's LLVM-based JIT for CoreCLR (2015) | **Dead**, archived, superseded | +| **Mono LLVM backend** | `--llvm` AOT path | Long-standing, production for AOT targets | +| **`rustc_codegen_clr`** | the *reverse* direction (Rust → CIL) | Referenced for context only; solves a different problem | + +**Burst is the honest precedent, and it teaches the real lesson**: the way to +get LLVM-grade codegen out of C# is not to translate all of C# — it is to carve +out a **restricted subset** where the .NET-specific invariants are cheap to +establish. Which is precisely Result 1, arrived at independently. + +## Is a PoC doable? Yes — and here is the ladder + +Rungs 1–3 are *already done and committed here*; the remaining cost is rung 4+. + +1. ✅ **Does LLVM help CIL-shaped loops at all?** — `run.sh` part 1. Answered: + only after frontend RCE. +2. ✅ **How big is the prize vs RyuJIT?** — 3.3–4.0× on numeric kernels, 1.00× + when memory-bound, 0× on pointer chasing. +3. ✅ **Is the prize reachable without LLVM?** — yes, hand-SIMD C# matches or + beats it. This is the rung that decides the project. +4. ⬜ **A real IL→LLVM translator** for a tiny opcode subset (`ldarg`/`ldloc`/ + `ldc.i4`/arithmetic/`ldelem.i4`/`stelem.i4`/`ldlen`/branches/`ret`), locals + as `alloca` + `mem2reg`. Perhaps 300–500 lines. **Only worth writing after + rung 3 says the automation is worth having** — and here it says it mostly + is not. +5. ⬜ Everything real: GC write barriers, exact stack maps for a moving GC, + exception semantics, P/Invoke, generics/shared code, tiering and OSR. This + is where LLILC died, and it is a compiler-team-years problem, not a weekend. + +**Recommendation: do not build rung 4+ here.** Not because it fails — it works — +but because rung 3 shows the payoff is available today in plain C#, and Result 1 +shows the enabling work carries no ownership content, so it would not compound +with anything Own.NET does. The experiment was worth running precisely because +it produced a clean negative on the part that looked most attractive. + +## What this does and does not reopen + +- **Does not reopen the `tech-debt-register.md` rejection of MLIR/LLVM.** That + rejection is about *OwnIR as a fact-interchange format* (§2: "instruction-level + compiler frameworks, wrong abstraction"). This note asks a different question — + LLVM as a *codegen backend for execution* — and independently lands on "no", + for different reasons. Both stand; neither is evidence for the other. +- **Does not create a work item.** Recorded as research context, per the + `research-landscape-2026.md` discipline: notes record, ROADMAP schedules. +- **Does leave one genuinely actionable thread**, and it is not the LLVM one: + the loop-invariant-expensive-call rule from Result 3. Filed here as an + observation, not a proposal — it would need corpus evidence (P-012) before + anyone writes a rule, and "expensive" is exactly the kind of word that + generates false positives. + +## Honest limits of this measurement + +- **One machine, one microarchitecture, 4 vCPU shared.** One K1 run showed + ~30% variance from a noisy neighbour; the reported medians are stable across + 3 runs but this is not a clean benchmarking rig. +- **Four kernels is not a corpus.** They were chosen to span straight-line + numeric / reduction / branchy / pointer-chasing, but no claim is made that + they represent real application profiles. +- **The C side is a *proxy* for CIL lowering, not real CIL.** It was hand-written + to mimic what a naive frontend emits. Rung 4 would replace the proxy with an + actual translator; until then, Result 1 is a statement about *loop shapes*, + which is how it is phrased. +- **No GC interaction is modelled at all.** The native kernels operate on pinned + buffers. Write barriers and stack maps — the things that actually killed + LLILC — are entirely absent from these numbers, so nothing here should be read + as a cost estimate for a real backend. From 5a7e4dc580a6f832261cc1b1c336675bdebefb83 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 05:58:06 +0000 Subject: [PATCH 2/4] docs(notes): measure the 1x-to-1092x spread that makes the runtime layer load-bearing Follow-up to llvm-codegen-feasibility.md, answering the objection that a static rule can reach the shape of a loop-invariant call but never its cost. Measured: four call shapes, each evaluated in-loop vs hoisted by hand, over collection sizes 4..100_000. Every row is the same syntactic shape, so a static rule matching that shape reports them identically. The penalty ranges from 1.0x (List.Count, an O(1) property -- a textbook false positive) to 1092x (OrderBy().First()). The decisive row is Any(x => x > threshold) at n=100_000: 8.3x when the predicate hits at element 0, 1025x when it never hits. The source is character-for-character identical; only the runtime value of threshold and the data distribution differ. No static analysis can separate those, and collection size does not predict cost either -- the short-circuiting row is flat across n while the other grows a hundredfold, so "flag only for large collections" is wrong in both directions. The conclusion is not that the static rule is worthless but that it and a profiler are each unactionable alone and complete each other: the profiler supplies magnitude without knowing the call is safely hoistable, the rule supplies the proof and the fix without knowing whether it buys 0% or 99.9%. That is Plan.md's existing Layer 1 -> Layer 2 shape, reusing the same confirmation pattern already used for subscription leaks, with a timing witness in place of a heap walk. It also repairs this note's parent, which flagged "expensive" as a false-positive generator: that holds for a static predicate, but measured it stops being a predicate and becomes a number. Also expands the Burst row of the landscape table into a full subsection, since the mechanism matters: Burst does not defeat the blockers found earlier, it defines them out of the language (no object header means no per-iteration ldlen; job-system bounds are loop-invariant by construction). Its value to us is cost calibration first, a real but partial design precedent second -- the parallel to OwnLang's buffer policies holds only for the restricted-language part, since the ownership content was measured at 0% -- and nothing as a component. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/invariant-cost-data/Program.cs | 123 ++++++++++++++++++ docs/notes/invariant-cost-data/linq.csproj | 8 ++ docs/notes/invariant-cost-data/run.sh | 11 ++ .../notes/invariant-cost-static-vs-runtime.md | 118 +++++++++++++++++ docs/notes/llvm-codegen-feasibility.md | 45 +++++++ 5 files changed, 305 insertions(+) create mode 100644 docs/notes/invariant-cost-data/Program.cs create mode 100644 docs/notes/invariant-cost-data/linq.csproj create mode 100755 docs/notes/invariant-cost-data/run.sh create mode 100644 docs/notes/invariant-cost-static-vs-runtime.md diff --git a/docs/notes/invariant-cost-data/Program.cs b/docs/notes/invariant-cost-data/Program.cs new file mode 100644 index 00000000..b5110421 --- /dev/null +++ b/docs/notes/invariant-cost-data/Program.cs @@ -0,0 +1,123 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Linq; +using System.Runtime.CompilerServices; + +// Question: a static rule can find "loop-invariant query evaluated inside a +// loop". Can it tell you how much it COSTS? Every case below is the SAME +// syntactic shape -- an invariant call in a loop body -- so a static rule sees +// them as identical. Measure the actual penalty. +static class Program +{ + static long sink; + + // ---- shape A: Any() over a List, predicate over a captured local ---- + [MethodImpl(MethodImplOptions.NoInlining)] + static long A_Inline(List data, int threshold, int iters) + { + long acc = 0; + for (int i = 0; i < iters; i++) + if (data.Any(x => x > threshold)) acc += i; // loop-invariant + return acc; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long A_Hoisted(List data, int threshold, int iters) + { + long acc = 0; + bool inv = data.Any(x => x > threshold); // hoisted by hand + for (int i = 0; i < iters; i++) if (inv) acc += i; + return acc; + } + + // ---- shape B: Count() on IEnumerable (O(n)) vs .Count property ---- + [MethodImpl(MethodImplOptions.NoInlining)] + static long B_Inline(IEnumerable data, int iters) + { + long acc = 0; + for (int i = 0; i < iters; i++) acc += data.Count(); + return acc; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long B_Hoisted(IEnumerable data, int iters) + { + long acc = 0; int c = data.Count(); + for (int i = 0; i < iters; i++) acc += c; + return acc; + } + + // ---- shape C: OrderBy().First() -- allocates + sorts every iteration ---- + [MethodImpl(MethodImplOptions.NoInlining)] + static long C_Inline(List data, int iters) + { + long acc = 0; + for (int i = 0; i < iters; i++) acc += data.OrderBy(x => x).First(); + return acc; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long C_Hoisted(List data, int iters) + { + long acc = 0; int v = data.OrderBy(x => x).First(); + for (int i = 0; i < iters; i++) acc += v; + return acc; + } + + // ---- shape D: a trivially cheap invariant -- the control ---- + [MethodImpl(MethodImplOptions.NoInlining)] + static long D_Inline(List data, int iters) + { + long acc = 0; + for (int i = 0; i < iters; i++) acc += data.Count; // O(1) property + return acc; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long D_Hoisted(List data, int iters) + { + long acc = 0; int c = data.Count; + for (int i = 0; i < iters; i++) acc += c; + return acc; + } + + static double Bench(Func f, int reps) + { + for (int i = 0; i < 20; i++) sink += f(); // warm + var sw = Stopwatch.StartNew(); + for (int i = 0; i < reps; i++) sink += f(); + sw.Stop(); + return sw.Elapsed.TotalMilliseconds / reps; + } + + static void Row(string shape, int n, Func inline, Func hoisted, int reps) + { + double ti = Bench(inline, reps), th = Bench(hoisted, reps); + long a = inline(), b = hoisted(); + string ok = a == b ? "" : $" !! MISMATCH {a} vs {b}"; + Console.WriteLine($"{shape,-28} n={n,-7} inline={ti,9:F4} ms hoisted={th,9:F4} ms penalty = {ti / th,8:F1}x{ok}"); + } + + static void Main() + { + const int Iters = 1000; + Console.WriteLine("Same syntactic shape everywhere: a loop-invariant call in a loop body."); + Console.WriteLine($"Inner loop = {Iters} iterations. 'penalty' = how much the un-hoisted version costs.\n"); + + foreach (int n in new[] { 4, 64, 4096, 100_000 }) + { + var list = Enumerable.Range(0, n).ToList(); + IEnumerable seq = list.Select(x => x); // hides ICollection fast path + int th = -1; // Any() hits on the FIRST element + + Row("A: Any(x => x > t) [hit@0]", n, () => A_Inline(list, th, Iters), () => A_Hoisted(list, th, Iters), 20); + Row("A: Any(x => x > t) [miss]", n, () => A_Inline(list, int.MaxValue, Iters), () => A_Hoisted(list, int.MaxValue, Iters), 5); + Row("B: Count() on IEnumerable", n, () => B_Inline(seq, Iters), () => B_Hoisted(seq, Iters), 5); + Row("C: OrderBy().First()", n, () => C_Inline(list, Iters), () => C_Hoisted(list, Iters), n > 10000 ? 1 : 3); + Row("D: .Count property (cheap)", n, () => D_Inline(list, Iters), () => D_Hoisted(list, Iters), 50); + Console.WriteLine(); + } + Console.WriteLine($"(sink={sink})"); + } +} diff --git a/docs/notes/invariant-cost-data/linq.csproj b/docs/notes/invariant-cost-data/linq.csproj new file mode 100644 index 00000000..2d643a65 --- /dev/null +++ b/docs/notes/invariant-cost-data/linq.csproj @@ -0,0 +1,8 @@ + + + Exe + net9.0 + disable + true + + diff --git a/docs/notes/invariant-cost-data/run.sh b/docs/notes/invariant-cost-data/run.sh new file mode 100755 index 00000000..f0f77e9f --- /dev/null +++ b/docs/notes/invariant-cost-data/run.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +# Reproduces the table in ../invariant-cost-static-vs-runtime.md. +# Requires the .NET SDK 9 (tested 9.0.316). +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +WORK="${1:-$(mktemp -d)}" +mkdir -p "$WORK"; cp "$HERE"/{Program.cs,linq.csproj} "$WORK/" +cd "$WORK" +dotnet build -c Release -v q --nologo +# steady state: measure tier-1 code, not the tier-0 warmup +DOTNET_TieredCompilation=0 dotnet bin/Release/net9.0/linq.dll diff --git a/docs/notes/invariant-cost-static-vs-runtime.md b/docs/notes/invariant-cost-static-vs-runtime.md new file mode 100644 index 00000000..49d0ef90 --- /dev/null +++ b/docs/notes/invariant-cost-static-vs-runtime.md @@ -0,0 +1,118 @@ +# A loop-invariant call costs between 1× and 1092× — and only runtime knows which + +Working note. **Trigger:** the follow-up to +[`llvm-codegen-feasibility.md`](llvm-codegen-feasibility.md). That note closed +with "loop-invariant expensive query inside a loop is a static-analysis target, +not a codegen target". The objection that followed is correct and is the reason +this note exists: + +> Statically we can reach the *shape*. But we will not know what actually +> happens at runtime — and that is the killer feature. RyuJIT may handle it +> fine, or it may not, as in the case where it was crawling. Static analysis +> says "this could be suboptimal"; only measurement says **how bad**. + +This note measures the spread, and it is wider than the objection assumed. +Harness in [`invariant-cost-data/`](invariant-cost-data/), `run.sh` reproduces. + +## Method + +Four call shapes, each written twice — evaluated inside the loop vs hoisted by +hand — over collection sizes 4 … 100 000, inner loop fixed at 1000 iterations. +`penalty = inline / hoisted`. Both variants are asserted to return the same +value. .NET 9.0.316, `DOTNET_TieredCompilation=0`, steady state. + +**Every row is the same syntactic shape**: a loop-invariant call in a loop body. +A static rule matching that shape sees all of them as one finding. + +## Result + +| shape | n=4 | n=64 | n=4 096 | n=100 000 | +|---|---|---|---|---| +| `A: Any(x => x > t)` — predicate hits at element 0 | 2.8× | 12.6× | 11.1× | **8.3×** | +| `A: Any(x => x > t)` — predicate never hits | 12.2× | 164× | 898× | **1025×** | +| `B: Count()` on `IEnumerable` | 53× | 347× | 936× | 1025× | +| `C: OrderBy().First()` | 362× | 822× | 1092× | 1003× | +| `D: .Count` property (O(1)) | 1.0× | 1.0× | 1.0×¹ | 1.0× | + +¹ one run showed 2.5× on sub-microsecond timings; that is measurement noise, not +an effect. + +## What the numbers actually say + +**1. The dynamic range is three orders of magnitude — 1.0× to 1092×.** A static +rule that reports all of these identically is not wrong, it is *useless for +prioritisation*. Row D is a textbook false positive: perfectly matching the +shape, costing exactly nothing, because `List.Count` is an O(1) property. + +**2. The killer row is A, and it settles the argument.** Compare the two `A` +rows at n=100 000: **8.3×** versus **1025×**. The source code is +**character-for-character identical** — `data.Any(x => x > threshold)`. The only +difference is the runtime *value* of `threshold` and the data distribution: +when the predicate hits at element 0, `Any` short-circuits and collection size +becomes irrelevant; when it never hits, the call is O(n) and the cost tracks n. + +No static analysis can separate those two, ever. Not a better checker, not +whole-program analysis — the two programs are the same program. The cost lives +entirely in data the analyzer does not have. + +**3. Collection size does not predict cost either.** Row A[hit] is *flat* across +n (2.8 → 12.6 → 11.1 → 8.3), while row A[miss] grows a hundredfold over the same +range. So even "flag it only for large collections" — the obvious heuristic +rescue — is wrong in both directions. + +## The consequence: this is Layer 1 → Layer 2, and we already have that shape + +The conclusion is not "the static rule is worthless". It is that **the static +rule and the profiler are each individually unactionable, and complete each +other exactly**: + +- **A profiler alone** tells you `data.Any(...)` is 30% of CPU. It does *not* + tell you the call is loop-invariant, i.e. that the fix is free and safe. Hot + is not the same as fixable, and most hot lines are hot because they do + necessary work. +- **A static rule alone** tells you the call is provably invariant and gives the + exact fix — hoist it. It cannot tell you whether that fix buys 0% or 99.9%, + so it cannot rank, and an unranked list at 1.0×-to-1092× precision is noise. +- **Together**: *"this line is 30% of CPU **and** it is provably loop-invariant + → hoist it, here is the edit."* That is a finding with a magnitude, a proof, + and a patch. Neither layer produces it alone. + +This is precisely the architecture [`Plan.md`](../../Plan.md) §1 already +describes — Layer 1 static → Layer 2 runtime → Layer 3 AI over finished +evidence — and precisely the confirmation pattern already used for subscription +leaks, where own-check flags and the runtime harness confirms retention +(`MED` static, `HIGH` once runtime-confirmed, Plan.md §2 category 2). The +performance case reuses the mechanism unchanged; only the witness differs +(a timer/profiler sample instead of a heap walk). + +It also **repairs the objection this note's parent raised against itself**. That +note called "expensive" the kind of word that generates false positives, and +that is right — *as a static predicate*. Measured, it stops being a predicate +and becomes a number. The runtime layer is what makes the static rule shippable, +not an optional enhancement to it. + +## Honest limits + +- **Microbenchmark, one machine.** These are isolated shapes with no cache + pressure from surrounding work; real applications will compress the extremes. + The *ordering* and the fact of the spread are the claim, not the exact + multipliers. +- **This is not evidence the rule is implementable.** Proving loop-invariance + for a LINQ chain requires proving the receiver, the captured locals, *and* the + lambda are all invariant across the loop body — that is real interprocedural + work (P-036 territory), not a syntactic match. Nothing here estimates that + cost. +- **`.NET 4.7.2` was not measured.** The trigger case was on Framework, where + there is no dynamic PGO and no guarded devirtualization; these .NET 9 numbers + are therefore a *lower* bound on the penalty, not a reproduction of it. +- **No corpus.** Four shapes chosen to span the range. Whether real code + clusters near 1× or near 1000× is unmeasured, and that distribution is what + would actually decide whether the rule is worth building. + +## Status + +**Recorded, not scheduled.** Notes record, the ROADMAP schedules. If this is +ever picked up, the honest first step is not the rule — it is +**mining (P-012) to find out whether the pattern occurs in real C# at all, and +with which cost distribution.** Building a checker for a pattern whose real-world +cost profile is unknown is how the 1.0× false positives get shipped. diff --git a/docs/notes/llvm-codegen-feasibility.md b/docs/notes/llvm-codegen-feasibility.md index 1a717b32..3353e309 100644 --- a/docs/notes/llvm-codegen-feasibility.md +++ b/docs/notes/llvm-codegen-feasibility.md @@ -204,6 +204,51 @@ get LLVM-grade codegen out of C# is not to translate all of C# — it is to carv out a **restricted subset** where the .NET-specific invariants are cheap to establish. Which is precisely Result 1, arrived at independently. +### What Burst actually is, and whether it helps us + +Burst compiles **HPC#** — "High-Performance C#", a deliberately crippled subset +— through LLVM instead of RyuJIT. The subset is the whole trick: no classes, no +managed heap references, no GC allocation, no exceptions in kernels; you work +over `struct`s and `NativeArray` inside Unity's job system, and you annotate +pointers `[NoAlias]`. Under those restrictions LLVM's auto-vectorizer, +unroller and inliner all apply, and Unity reports 10–100× on compute-heavy +kernels. + +Note *why* those restrictions matter, against Result 1: forbidding managed +references and using `NativeArray` removes the object header, so there is no +per-iteration `ldlen`; the job system's bounds are loop-invariant by +construction, so the range checks hoist. **Burst does not beat the blockers +found above — it defines them out of the language.** That is the entire +mechanism. + +Three honest conclusions about how it helps us, in decreasing order of value: + +1. **As a cost calibration — the main value.** Burst is what "success" costs. + A funded team, a decade, and a language subset severe enough that ordinary + C# does not compile. It converts "could we do this?" into "this is the size + of the thing", which is why rung 4+ is not recommended. +2. **As a design precedent, with a real but partial parallel here.** OwnLang is + already a restricted language with storage discipline enforced by + construction ([`spec/BufferPolicies.md`](../../spec/BufferPolicies.md): + `stack`/`scratch`/`pooled`/`native`, where B1 forbids stack buffers from + escaping). So the structural analogy `Burst : HPC#` ≈ + `LLVM backend : OwnSharp subset` is not fantasy — a language that already + pins storage and escape could supply the invariants a frontend needs. + **But** the parallel stops exactly where Result 2 does: the invariants worth + supplying are range-check elimination and length immutability, and the + ownership content contributes **0%**. OwnLang would be helping as *a + restricted language*, not as *an ownership system* — so this is not a reason + to build it, and it does not make the ownership work pay off in codegen. +3. **Not usable as a component.** Burst is Unity-coupled — the package, the job + system, `NativeArray`. There is nothing to vendor or reuse outside Unity. + +One transferable practice, independent of any of the above: Burst ships an +**inspector** showing the generated assembly per kernel, because even inside the +subset, whether a loop vectorised is not predictable from the source. That is +the same "measure, do not assume" discipline this note's `-Rpass` output and +`DOTNET_JitDisasm` dumps applied — and the same lesson as +[`invariant-cost-static-vs-runtime.md`](invariant-cost-static-vs-runtime.md). + ## Is a PoC doable? Yes — and here is the ladder Rungs 1–3 are *already done and committed here*; the remaining cost is rung 4+. From c6578d2fb7a9127a84df22962dc3bbb0ce86fbd4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 07:07:18 +0000 Subject: [PATCH 3/4] docs(notes): name the missing summary domain precisely The invariant-cost note filed a LINQ loop-invariance rule under "P-036 territory". P-036 is the right host for the machinery (call graph, MethodSummary, SCC composition), but none of its five summary domains -- ownership, obligation, progress, region, task -- is a purity/effect-freedom domain, and effects are owned by P-008, which is explicitly horizon. A reader would otherwise open P-036 expecting to find the domain there. Also records that P-036's unknown/external-call policy would classify Any(userLambda) as unresolved or unsupported, so under its own rules the static half yields a candidate with declared uncertainty rather than a verdict -- independently reaching the note's conclusion that magnitude comes from the runtime layer. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/invariant-cost-static-vs-runtime.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/docs/notes/invariant-cost-static-vs-runtime.md b/docs/notes/invariant-cost-static-vs-runtime.md index 49d0ef90..641e9e4a 100644 --- a/docs/notes/invariant-cost-static-vs-runtime.md +++ b/docs/notes/invariant-cost-static-vs-runtime.md @@ -99,9 +99,20 @@ not an optional enhancement to it. multipliers. - **This is not evidence the rule is implementable.** Proving loop-invariance for a LINQ chain requires proving the receiver, the captured locals, *and* the - lambda are all invariant across the loop body — that is real interprocedural - work (P-036 territory), not a syntactic match. Nothing here estimates that - cost. + lambda are all effect-free across the loop body — real interprocedural work, + not a syntactic match. [P-036](../proposals/P-036-interprocedural-semantic-architecture.md) + is the right *host* for it (call graph, `MethodSummary`, SCC composition), but + **none of its five summary domains** — ownership, obligation, progress, + region, task — is a purity/effect-freedom domain, so this would be a sixth + one, and effects are owned by [P-008](../proposals/P-008-effects-and-resources.md) + (draft, explicitly horizon). Nothing here estimates that cost. +- **P-036's own rules would refuse the claim by default.** Its unknown/external + call policy classifies `Any(userLambda)` as an unresolved or unsupported + target: conservative defaults, recorded degraded precision, and the standing + rule that a check "may not pretend the call was proven harmless". So the + static half would emit a *candidate with declared uncertainty*, not a verdict + — which is another way of arriving at the same conclusion as the body of this + note: the magnitude has to come from the runtime layer. - **`.NET 4.7.2` was not measured.** The trigger case was on Framework, where there is no dynamic PGO and no guarded devirtualization; these .NET 9 numbers are therefore a *lower* bound on the penalty, not a reproduction of it. From 0cd431136969ee062a5368ee9fc9d303a5d3ea5b Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 17:44:56 +0000 Subject: [PATCH 4/4] =?UTF-8?q?docs:=20address=20review=20=E2=80=94=20scop?= =?UTF-8?q?e=20the=20claims=20to=20what=20was=20measured,=20and=20make=20t?= =?UTF-8?q?he=20gate=20fail?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verified each finding before acting; all are accuracy fixes to my own notes and harnesses, and none changes a measured result. Harness correctness. The invariant-cost Row() only PRINTED a mismatch, so a non-equivalent pair would still exit 0 and be reported as a measurement -- it now checks before timing and throws, which also makes the note's "asserted" wording true. The K3 gate compared the five variants only to each other, where a shared defect passes; it now also pins the expected constant 25830282. run.sh no longer leaks the mktemp workdir it creates (trap on EXIT, only when it owns the directory) and prints the SDK and runtime it actually selected instead of merely claiming 9.0.316. Overclaims scoped to the evidence. "No static analysis can separate those two, ever" was too strong: Main builds data with a deterministic Enumerable.Range and passes constant thresholds, so a whole-program analyzer could constant-propagate these particular sites. The claim is now about a local rule and about general unknown inputs, which is the case a real rule faces. "Costing exactly nothing" and "useless for prioritisation" become no measurable penalty in this run and cannot rank this sample without runtime data. The .NET 9 numbers are no longer called a lower bound for 4.7.2 -- different JIT and different LINQ, so they are simply not comparable without a matched Framework measurement. LOH guidance corrected in two ways: 85,000 is the default, movable by System.GC.LOHThreshold and DOTNET_GCLOHThreshold, and the 24-byte overhead is a CoreCLR-x64 detail rather than a portable constant, so the portable advice is to measure. The GC probe now carries the caveat that CollectionCount and GetGCMemoryInfo are process-wide, moved by background and parallel GC, with GetGCMemoryInfo zero-valued at Index 0 when no collection of that kind occurred. Labels made accurate: five K3 variants rather than four, Count() on a Select-wrapped sequence rather than all IEnumerable, OrderBy().First() as repeated linear key scanning via .NET 9's TryGetFirst rather than a full sort per iteration, and DOTNET_TieredCompilation=0 described as non-tiered FullOpts rather than tier 1. Both harnesses re-run clean: the vectorization matrix is unchanged and every reported number reproduces. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PbyzyVi7fibuSKLweuXgea --- docs/notes/gc-observability-and-loh.md | 41 ++++++++++++++----- docs/notes/invariant-cost-data/Program.cs | 20 ++++++--- docs/notes/invariant-cost-data/run.sh | 25 +++++++++-- .../notes/invariant-cost-static-vs-runtime.md | 36 ++++++++++------ .../llvm-codegen-feasibility-data/Kernels.cs | 10 +++-- .../llvm-codegen-feasibility-data/README.md | 12 ++++-- docs/notes/llvm-codegen-feasibility.md | 19 +++++---- 7 files changed, 118 insertions(+), 45 deletions(-) diff --git a/docs/notes/gc-observability-and-loh.md b/docs/notes/gc-observability-and-loh.md index cb96f53e..a381f946 100644 --- a/docs/notes/gc-observability-and-loh.md +++ b/docs/notes/gc-observability-and-loh.md @@ -13,19 +13,32 @@ complement" argument this reuses verbatim. ## The one factual correction worth taking Our docs treat the Large Object Heap threshold as **85,000 bytes** of payload. -That is the documented constant, but it is not the predicate. The GC compares -the **full object size** — payload + object header + method table pointer + -alignment padding — so an array whose *payload* is comfortably under the limit -still lands on the LOH: +That is the documented **default** constant, but it is not the predicate. Two +things are wrong with the folklore reading: -> `byte[84_999]` → 85,023 bytes with the 24-byte array header → rounds to -> 85,024 → **allocated on the LOH.** +1. **The threshold is configurable.** `System.GC.LOHThreshold` + (`runtimeconfig.json`) and `DOTNET_GCLOHThreshold` (environment, hex) raise + it, so 85,000 describes a default configuration, not a law. +2. **The comparison is against full object size**, not payload — payload + + object header + method-table pointer + the array length field + alignment + padding — so an array whose *payload* sits comfortably under the limit can + still land on the LOH. + +Worked on the environment this was checked against (**.NET CoreCLR, x64, +default GC configuration**), where that overhead is 24 bytes: + +> `byte[84_999]` → 85,023 bytes → rounds to 85,024 → **allocated on the LOH.** + +The header size is a **platform and runtime detail**, not a portable constant — +object layout and alignment differ by architecture and runtime version, so the +24 above should not be copied into another context as a given. The practical consequence, and the reason it is worth writing down: the familiar "keep buffers under 85,000" folklore is **off by roughly one header**, and a -pool sized to exactly `85_000 - 1` is on the wrong heap. Anyone sizing a buffer -pool against the constant should size against `85_000 - 24 - padding`, or -simply measure. +pool sized to exactly `85_000 - 1` is on the wrong heap under the default +configuration. The portable advice is not a corrected arithmetic constant — it +is **measure on your target runtime**, since both the threshold and the overhead +can move. Where this touches our docs: [`ROADMAP.md`](../ROADMAP.md) and [`Plan.md`](../../Plan.md) both list LOH fragmentation in the detectability @@ -55,7 +68,15 @@ Today our runtime layer ([`Plan.md`](../../Plan.md) §2, category 12) routes badly overweight for **orientation** — it means no GC fact can be established in CI, on Linux, or in a unit test. A `GC.GetGCMemoryInfo` / `GC.CollectionCount(n)` snapshot around a scenario costs nothing and needs no -stand. +stand — with one caveat that has to travel with it: **both are process-wide, not +scenario-scoped.** `CollectionCount(n)` counts every collection since process +start (and a higher-generation collection bumps the lower ones too); +`GetGCMemoryInfo()` describes the *last* collection and returns an all-zero +struct with `Index == 0` when none of the requested kind has happened. Parallel +tests or background GC move both from outside the scenario. So a probe used as +an assertion has to record counts at the scenario boundary and compare +`GetGCMemoryInfo()` only when the GC index matches — otherwise it is a heuristic +wearing an assertion's clothes. So: a cheap in-test GC probe is a reasonable sibling to P-034's disposal quarantine, under the same honest caveat P-034 already states — it proves *what diff --git a/docs/notes/invariant-cost-data/Program.cs b/docs/notes/invariant-cost-data/Program.cs index b5110421..0dab7eca 100644 --- a/docs/notes/invariant-cost-data/Program.cs +++ b/docs/notes/invariant-cost-data/Program.cs @@ -31,7 +31,8 @@ static long A_Hoisted(List data, int threshold, int iters) return acc; } - // ---- shape B: Count() on IEnumerable (O(n)) vs .Count property ---- + // ---- shape B: Count() on a Select-wrapped sequence (O(n) here; Count() + // is O(1) when the source implements ICollection) vs .Count property ---- [MethodImpl(MethodImplOptions.NoInlining)] static long B_Inline(IEnumerable data, int iters) { @@ -48,7 +49,9 @@ static long B_Hoisted(IEnumerable data, int iters) return acc; } - // ---- shape C: OrderBy().First() -- allocates + sorts every iteration ---- + // ---- shape C: OrderBy().First() -- repeated linear key scanning every + // iteration (.NET 9 takes a specialized TryGetFirst path rather than + // materializing and sorting a buffer) ---- [MethodImpl(MethodImplOptions.NoInlining)] static long C_Inline(List data, int iters) { @@ -93,10 +96,15 @@ static double Bench(Func f, int reps) static void Row(string shape, int n, Func inline, Func hoisted, int reps) { - double ti = Bench(inline, reps), th = Bench(hoisted, reps); + // Correctness gate FIRST, and it must fail the run: timing two + // non-equivalent implementations produces a meaningless ratio, and a + // printed warning would still exit 0 and look like a good measurement. long a = inline(), b = hoisted(); - string ok = a == b ? "" : $" !! MISMATCH {a} vs {b}"; - Console.WriteLine($"{shape,-28} n={n,-7} inline={ti,9:F4} ms hoisted={th,9:F4} ms penalty = {ti / th,8:F1}x{ok}"); + if (a != b) + throw new InvalidOperationException($"{shape} n={n}: MISMATCH {a} vs {b}"); + + double ti = Bench(inline, reps), th = Bench(hoisted, reps); + Console.WriteLine($"{shape,-38} n={n,-7} inline={ti,9:F4} ms hoisted={th,9:F4} ms penalty = {ti / th,8:F1}x"); } static void Main() @@ -113,7 +121,7 @@ static void Main() Row("A: Any(x => x > t) [hit@0]", n, () => A_Inline(list, th, Iters), () => A_Hoisted(list, th, Iters), 20); Row("A: Any(x => x > t) [miss]", n, () => A_Inline(list, int.MaxValue, Iters), () => A_Hoisted(list, int.MaxValue, Iters), 5); - Row("B: Count() on IEnumerable", n, () => B_Inline(seq, Iters), () => B_Hoisted(seq, Iters), 5); + Row("B: Count() on Select-wrapped IEnumerable", n, () => B_Inline(seq, Iters), () => B_Hoisted(seq, Iters), 5); Row("C: OrderBy().First()", n, () => C_Inline(list, Iters), () => C_Hoisted(list, Iters), n > 10000 ? 1 : 3); Row("D: .Count property (cheap)", n, () => D_Inline(list, Iters), () => D_Hoisted(list, Iters), 50); Console.WriteLine(); diff --git a/docs/notes/invariant-cost-data/run.sh b/docs/notes/invariant-cost-data/run.sh index f0f77e9f..34e5fcea 100755 --- a/docs/notes/invariant-cost-data/run.sh +++ b/docs/notes/invariant-cost-data/run.sh @@ -1,11 +1,30 @@ #!/usr/bin/env bash # Reproduces the table in ../invariant-cost-static-vs-runtime.md. -# Requires the .NET SDK 9 (tested 9.0.316). +# Requires a .NET 9 SDK. The published numbers were taken on 9.0.316; any 9.0.x +# should reproduce the ORDER and the spread, but exact multipliers will differ +# by machine and patch level -- the selected SDK/runtime is printed below so a +# rerun is self-describing rather than merely claiming reproduction. set -euo pipefail HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -WORK="${1:-$(mktemp -d)}" + +# Only clean up a workdir we created ourselves; never delete a caller's. +if [[ $# -ge 1 ]]; then + WORK="$1" +else + WORK="$(mktemp -d)" + trap 'rm -rf -- "$WORK"' EXIT +fi + mkdir -p "$WORK"; cp "$HERE"/{Program.cs,linq.csproj} "$WORK/" cd "$WORK" + +echo "== toolchain actually used ==" +dotnet --version +dotnet --list-runtimes | grep -E '^Microsoft\.NETCore\.App 9\.0\.' || true +echo + dotnet build -c Release -v q --nologo -# steady state: measure tier-1 code, not the tier-0 warmup +# Tiering disabled: every method is compiled straight to FullOpts, so the +# measurement is steady-state code. (This is non-tiered mode -- "tier 1" is a +# tiering concept and does not apply when tiering is off.) DOTNET_TieredCompilation=0 dotnet bin/Release/net9.0/linq.dll diff --git a/docs/notes/invariant-cost-static-vs-runtime.md b/docs/notes/invariant-cost-static-vs-runtime.md index 641e9e4a..bdc19de1 100644 --- a/docs/notes/invariant-cost-static-vs-runtime.md +++ b/docs/notes/invariant-cost-static-vs-runtime.md @@ -18,8 +18,11 @@ Harness in [`invariant-cost-data/`](invariant-cost-data/), `run.sh` reproduces. Four call shapes, each written twice — evaluated inside the loop vs hoisted by hand — over collection sizes 4 … 100 000, inner loop fixed at 1000 iterations. -`penalty = inline / hoisted`. Both variants are asserted to return the same -value. .NET 9.0.316, `DOTNET_TieredCompilation=0`, steady state. +`penalty = inline / hoisted`. The harness asserts both variants return the same +value and **throws** if they do not, so a non-equivalent pair cannot be reported +as a measurement. .NET 9.0.316, `DOTNET_TieredCompilation=0` — tiering off, so +every method is compiled straight to FullOpts (non-tiered mode; "tier 1" is a +tiering concept that does not apply when tiering is disabled). **Every row is the same syntactic shape**: a loop-invariant call in a loop body. A static rule matching that shape sees all of them as one finding. @@ -30,7 +33,7 @@ A static rule matching that shape sees all of them as one finding. |---|---|---|---|---| | `A: Any(x => x > t)` — predicate hits at element 0 | 2.8× | 12.6× | 11.1× | **8.3×** | | `A: Any(x => x > t)` — predicate never hits | 12.2× | 164× | 898× | **1025×** | -| `B: Count()` on `IEnumerable` | 53× | 347× | 936× | 1025× | +| `B: Count()` on a `Select`-wrapped sequence | 53× | 347× | 936× | 1025× | | `C: OrderBy().First()` | 362× | 822× | 1092× | 1003× | | `D: .Count` property (O(1)) | 1.0× | 1.0× | 1.0×¹ | 1.0× | @@ -40,9 +43,10 @@ an effect. ## What the numbers actually say **1. The dynamic range is three orders of magnitude — 1.0× to 1092×.** A static -rule that reports all of these identically is not wrong, it is *useless for -prioritisation*. Row D is a textbook false positive: perfectly matching the -shape, costing exactly nothing, because `List.Count` is an O(1) property. +rule that reports all of these identically is not wrong, but *it cannot rank +this sample without runtime data*. Row D matches the shape perfectly and shows +**no measurable penalty in this run** — `List.Count` is an O(1) property — so on +this evidence it is the false-positive shape. **2. The killer row is A, and it settles the argument.** Compare the two `A` rows at n=100 000: **8.3×** versus **1025×**. The source code is @@ -51,9 +55,15 @@ difference is the runtime *value* of `threshold` and the data distribution: when the predicate hits at element 0, `Any` short-circuits and collection size becomes irrelevant; when it never hits, the call is O(n) and the cost tracks n. -No static analysis can separate those two, ever. Not a better checker, not -whole-program analysis — the two programs are the same program. The cost lives -entirely in data the analyzer does not have. +A **local** rule — one reasoning about the call site without runtime or +call-site values — cannot separate those two, because at that site the two +programs *are* the same program. The honest scope: in this harness `Main` builds +its data with a deterministic `Enumerable.Range` and passes constant thresholds, +so a whole-program analyzer could in principle constant-propagate these +particular call sites. What it could not do is predict the cost for **general, +unknown inputs** — a `threshold` from configuration, a collection from a +database — which is the case the rule would actually face. The cost lives in +data the analyzer does not have. **3. Collection size does not predict cost either.** Row A[hit] is *flat* across n (2.8 → 12.6 → 11.1 → 8.3), while row A[miss] grows a hundredfold over the same @@ -113,9 +123,11 @@ not an optional enhancement to it. static half would emit a *candidate with declared uncertainty*, not a verdict — which is another way of arriving at the same conclusion as the body of this note: the magnitude has to come from the runtime layer. -- **`.NET 4.7.2` was not measured.** The trigger case was on Framework, where - there is no dynamic PGO and no guarded devirtualization; these .NET 9 numbers - are therefore a *lower* bound on the penalty, not a reproduction of it. +- **`.NET 4.7.2` was not measured, and these numbers are not comparable to it.** + The trigger case was on Framework, which has a different JIT and a different + LINQ implementation; no lower-bound (or upper-bound) inference about Framework + follows from a .NET 9 run. Claiming one would need a matched Framework + measurement, which this harness does not do. - **No corpus.** Four shapes chosen to span the range. Whether real code clusters near 1× or near 1000× is unmeasured, and that distribution is what would actually decide whether the rule is worth building. diff --git a/docs/notes/llvm-codegen-feasibility-data/Kernels.cs b/docs/notes/llvm-codegen-feasibility-data/Kernels.cs index 71a93697..ee075ddb 100644 --- a/docs/notes/llvm-codegen-feasibility-data/Kernels.cs +++ b/docs/notes/llvm-codegen-feasibility-data/Kernels.cs @@ -132,12 +132,16 @@ public static unsafe void Run() t = Time(() => sink += SumFree(p, N, N), Reps); Console.WriteLine($"LLVM -O3 no check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k2m,5:F2}x"); - // correctness gate: every K3 variant must return the SAME value. + // correctness gate: every K3 variant must return the SAME value, and + // that value must be the pinned expected result -- comparing the + // variants only to each other would let a shared defect pass. + const long ExpectedFilterSum = 25_830_282; long r0 = FilterSumManaged(a, 200, 800), r1 = FilterSumBranchless(a, 200, 800), r2 = FilterSumSimd(a, 200, 800), r3 = FilterSumNative(p, N, N, 200, 800); long r4 = FilterSumFree(p, N, N, 200, 800); - if (r0 != r1 || r0 != r2 || r0 != r3 || r0 != r4) - throw new Exception($"K3 MISMATCH scalar={r0} branchless={r1} simd={r2} native={r3} free={r4}"); + if (r0 != ExpectedFilterSum || r0 != r1 || r0 != r2 || r0 != r3 || r0 != r4) + throw new Exception($"K3 MISMATCH expected={ExpectedFilterSum} scalar={r0} " + + $"branchless={r1} simd={r2} native={r3} free={r4}"); long q0 = SumManaged(a), q1 = SumNative(p, N, N), q2 = SumFree(p, N, N); if (q0 != q1 || q0 != q2) throw new Exception($"K2 MISMATCH {q0} vs {q1} vs {q2}"); Console.WriteLine($"\n-- K3 data-dependent branch (filter+sum) -- [all variants agree: {r0}]"); diff --git a/docs/notes/llvm-codegen-feasibility-data/README.md b/docs/notes/llvm-codegen-feasibility-data/README.md index d914e0f5..0a110f7a 100644 --- a/docs/notes/llvm-codegen-feasibility-data/README.md +++ b/docs/notes/llvm-codegen-feasibility-data/README.md @@ -15,12 +15,16 @@ $ ./run.sh [workdir] | `native.c` | K1 (`dst[i] = a[i] + b[i]*3`), noalias and aliasing variants | | `native2.c` | K2 reduction, K3 data-dependent branch, K4 pointer chase | | `Program.cs` | K1 harness: RyuJIT scalar / `Vector256` / `Vector256` x4 / native | -| `Kernels.cs` | K2-K4 harness, incl. the correctness gate across all K3 variants | +| `Kernels.cs` | K2-K4 harness, incl. the correctness gate across all five K3 variants | Two things the harness enforces on purpose, both because they were got wrong first (see "Two measurement traps" in the note): -- **`DOTNET_TieredCompilation=0` plus a 5000-call warmup.** A short warmup - measures partly-tier-0 code and inflates LLVM's advantage by ~40%. -- **A correctness gate.** All K3 variants must return `25830282`; an unequal +- **`DOTNET_TieredCompilation=0` plus a 5000-call warmup.** Tiering off means + everything compiles straight to FullOpts (non-tiered mode). A short warmup + under the tiered default measures partly-tier-0 code and inflates LLVM's + advantage by ~40%. +- **A correctness gate.** All **five** K3 variants must agree *and* equal the + pinned constant `25830282` (comparing them only to each other would let a + shared defect through); an unequal accumulator width silently makes one variant do less work. diff --git a/docs/notes/llvm-codegen-feasibility.md b/docs/notes/llvm-codegen-feasibility.md index 3353e309..18f01dbb 100644 --- a/docs/notes/llvm-codegen-feasibility.md +++ b/docs/notes/llvm-codegen-feasibility.md @@ -41,13 +41,18 @@ and reproduced by `run.sh`, so every number below is falsifiable. - **Controls are the point.** Every kernel carries a hand-written C# SIMD variant, because "LLVM beats scalar C#" is a boring claim; "LLVM beats *well-written* C#" is the claim that would justify building anything. -- **Correctness gate.** All four K3 variants (scalar, branchless, hand-SIMD, - native) must return the identical value or the harness throws. They agree on - `25830282`. An early version of the SIMD control accumulated in 32-bit while - the scalar accumulated in 64-bit — it produced a flattering **33×** that was - simply *less work*. The gate exists because that mistake was made here. -- **Steady state.** Reported numbers use `DOTNET_TieredCompilation=0` plus a - 5000-call warmup. See the measurement traps below — this is not a detail. +- **Correctness gate.** All **five** K3 variants — scalar, branchless, + hand-SIMD, native with the per-element check, and native with the check + hoisted — must return the identical value, and that value must equal the + pinned constant `25830282`, or the harness throws. An early version of the + SIMD control accumulated in 32-bit while the scalar accumulated in 64-bit — + it produced a flattering **33×** that was simply *less work*. The gate exists + because that mistake was made here. +- **Steady state.** Reported numbers disable tiered compilation + (`DOTNET_TieredCompilation=0`, i.e. everything is compiled straight to + FullOpts — not "tier 1", which is a tiering concept) and additionally warm up + 5000 calls so the tiered default is also at steady state. See the measurement + traps below — this is not a detail. ### Two measurement traps (both hit during this work)