diff --git a/docs/notes/gc-observability-and-loh.md b/docs/notes/gc-observability-and-loh.md new file mode 100644 index 00000000..a381f946 --- /dev/null +++ b/docs/notes/gc-observability-and-loh.md @@ -0,0 +1,120 @@ +# GC observability and the LOH — what is worth adopting, and what stays runtime-only + +Working note. **Trigger:** the *GCExperiment* write-up ("Making .NET GC +behaviour observable"), proposed as material to fold into our runtime layer. +This note records what is genuinely new in it, corrects one number our own docs +imply, and pins the boundary that must not move. + +Companion to [`llvm-codegen-feasibility.md`](llvm-codegen-feasibility.md) (the +other half of the same discussion) and to +[P-034](../proposals/P-034-runtime-lifetime-guard.md), whose "ClrMD-free +complement" argument this reuses verbatim. + +## The one factual correction worth taking + +Our docs treat the Large Object Heap threshold as **85,000 bytes** of payload. +That is the documented **default** constant, but it is not the predicate. Two +things are wrong with the folklore reading: + +1. **The threshold is configurable.** `System.GC.LOHThreshold` + (`runtimeconfig.json`) and `DOTNET_GCLOHThreshold` (environment, hex) raise + it, so 85,000 describes a default configuration, not a law. +2. **The comparison is against full object size**, not payload — payload + + object header + method-table pointer + the array length field + alignment + padding — so an array whose *payload* sits comfortably under the limit can + still land on the LOH. + +Worked on the environment this was checked against (**.NET CoreCLR, x64, +default GC configuration**), where that overhead is 24 bytes: + +> `byte[84_999]` → 85,023 bytes → rounds to 85,024 → **allocated on the LOH.** + +The header size is a **platform and runtime detail**, not a portable constant — +object layout and alignment differ by architecture and runtime version, so the +24 above should not be copied into another context as a given. + +The practical consequence, and the reason it is worth writing down: the familiar +"keep buffers under 85,000" folklore is **off by roughly one header**, and a +pool sized to exactly `85_000 - 1` is on the wrong heap under the default +configuration. The portable advice is not a corrected arithmetic constant — it +is **measure on your target runtime**, since both the threshold and the overhead +can move. + +Where this touches our docs: [`ROADMAP.md`](../ROADMAP.md) and +[`Plan.md`](../../Plan.md) both list LOH fragmentation in the detectability +matrix. Neither states a threshold, so **neither is wrong** — but if a threshold +is ever quoted in a rule, a diagnostic message, or a talk, it must be the +full-object-size version, not the payload one. + +## What GCExperiment is, and what is actually adoptable + +Four self-contained experiments (LOH placement; generation promotion; +allocation pressure; LOH fragmentation) built on ordinary public APIs — +`GC.GetGeneration`, `GC.Collect` with forced modes, +`GC.WaitForPendingFinalizers`, `GCSettings.LargeObjectHeapCompactionMode`, +plus small `GCMonitor`/`GCInfo` helpers for snapshots and size estimation. It +also flags a real measurement trap: without `GC.KeepAlive`, the JIT can shorten +an object's lifetime and skew the result. + +The adoptable idea is **not** the GC content, which is well-trodden. It is the +**delivery shape**, and it is the same shape P-034 already argued for from a +different direction: + +> a lifetime/GC observation that runs in an ordinary `dotnet test`, on any OS, +> with no PerfView, no ETW, no Windows stand, and no ClrMD heap walk. + +Today our runtime layer ([`Plan.md`](../../Plan.md) §2, category 12) routes +*everything* GC-shaped to PerfView + ETW. That is correct for **evidence** and +badly overweight for **orientation** — it means no GC fact can be established in +CI, on Linux, or in a unit test. A `GC.GetGCMemoryInfo` / +`GC.CollectionCount(n)` snapshot around a scenario costs nothing and needs no +stand — with one caveat that has to travel with it: **both are process-wide, not +scenario-scoped.** `CollectionCount(n)` counts every collection since process +start (and a higher-generation collection bumps the lower ones too); +`GetGCMemoryInfo()` describes the *last* collection and returns an all-zero +struct with `Index == 0` when none of the requested kind has happened. Parallel +tests or background GC move both from outside the scenario. So a probe used as +an assertion has to record counts at the scenario boundary and compare +`GetGCMemoryInfo()` only when the GC index matches — otherwise it is a heuristic +wearing an assertion's clothes. + +So: a cheap in-test GC probe is a reasonable sibling to P-034's disposal +quarantine, under the same honest caveat P-034 already states — it proves *what +the counters did during this test*, bounded by test coverage, and it is blind to +*why*. It is a debug assertion, not an auditor. + +## The boundary that does not move + +**None of this makes LOH fragmentation statically detectable.** The matrix rows +stay exactly as written: + +- `ROADMAP.md`: LOH fragmentation → ❌ **impossible** (depends on runtime data + volume / GC timing) +- `Plan.md` category 12: heavy dictionaries / LOH fragmentation / Gen2 bloat → + **impossible** static → **RUNTIME** + +A GC probe is a **runtime witness**, and it lives on the runtime side of the +line the detectability matrix draws. The matrix exists specifically to stop +runtime-shaped problems from being hung on a static checker +([`Plan.md`](../../Plan.md) §1: it "*forbids*" exactly that, killing a class of +false positives in advance). Making GC behaviour cheaper to *observe* is not an +argument for making it *inferred*, and this note must not be cited as one. + +The one thing genuinely on the static side is unchanged and already ours: the +`ArrayPool`/`Span` misuse family (`POOL001`–`003`, P-007). Rent/return balance +is structurally visible; fragmentation is not. + +## Status + +**Recorded, not scheduled.** Per the `research-landscape-2026.md` discipline, +notes record and the ROADMAP schedules. Concretely, if anything is ever picked +up from here: + +1. **The LOH threshold correction** — free, and the only item with a + correctness argument behind it. Applies wherever a number gets quoted. +2. **A GC-counter probe as a P-034 sibling** — small, attaches to the open + question P-034 already asks (a new `Own.Diagnostics` package vs living in + OwnAudit's `runtime/`). Do not file it separately; it is the same decision. +3. **Nothing else.** The generation-promotion and allocation-pressure + experiments are educational rather than diagnostic, and we do not need to + re-derive published GC behaviour to ship a checker. diff --git a/docs/notes/invariant-cost-data/Program.cs b/docs/notes/invariant-cost-data/Program.cs new file mode 100644 index 00000000..0dab7eca --- /dev/null +++ b/docs/notes/invariant-cost-data/Program.cs @@ -0,0 +1,131 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Linq; +using System.Runtime.CompilerServices; + +// Question: a static rule can find "loop-invariant query evaluated inside a +// loop". Can it tell you how much it COSTS? Every case below is the SAME +// syntactic shape -- an invariant call in a loop body -- so a static rule sees +// them as identical. Measure the actual penalty. +static class Program +{ + static long sink; + + // ---- shape A: Any() over a List, predicate over a captured local ---- + [MethodImpl(MethodImplOptions.NoInlining)] + static long A_Inline(List data, int threshold, int iters) + { + long acc = 0; + for (int i = 0; i < iters; i++) + if (data.Any(x => x > threshold)) acc += i; // loop-invariant + return acc; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long A_Hoisted(List data, int threshold, int iters) + { + long acc = 0; + bool inv = data.Any(x => x > threshold); // hoisted by hand + for (int i = 0; i < iters; i++) if (inv) acc += i; + return acc; + } + + // ---- shape B: Count() on a Select-wrapped sequence (O(n) here; Count() + // is O(1) when the source implements ICollection) vs .Count property ---- + [MethodImpl(MethodImplOptions.NoInlining)] + static long B_Inline(IEnumerable data, int iters) + { + long acc = 0; + for (int i = 0; i < iters; i++) acc += data.Count(); + return acc; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long B_Hoisted(IEnumerable data, int iters) + { + long acc = 0; int c = data.Count(); + for (int i = 0; i < iters; i++) acc += c; + return acc; + } + + // ---- shape C: OrderBy().First() -- repeated linear key scanning every + // iteration (.NET 9 takes a specialized TryGetFirst path rather than + // materializing and sorting a buffer) ---- + [MethodImpl(MethodImplOptions.NoInlining)] + static long C_Inline(List data, int iters) + { + long acc = 0; + for (int i = 0; i < iters; i++) acc += data.OrderBy(x => x).First(); + return acc; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long C_Hoisted(List data, int iters) + { + long acc = 0; int v = data.OrderBy(x => x).First(); + for (int i = 0; i < iters; i++) acc += v; + return acc; + } + + // ---- shape D: a trivially cheap invariant -- the control ---- + [MethodImpl(MethodImplOptions.NoInlining)] + static long D_Inline(List data, int iters) + { + long acc = 0; + for (int i = 0; i < iters; i++) acc += data.Count; // O(1) property + return acc; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long D_Hoisted(List data, int iters) + { + long acc = 0; int c = data.Count; + for (int i = 0; i < iters; i++) acc += c; + return acc; + } + + static double Bench(Func f, int reps) + { + for (int i = 0; i < 20; i++) sink += f(); // warm + var sw = Stopwatch.StartNew(); + for (int i = 0; i < reps; i++) sink += f(); + sw.Stop(); + return sw.Elapsed.TotalMilliseconds / reps; + } + + static void Row(string shape, int n, Func inline, Func hoisted, int reps) + { + // Correctness gate FIRST, and it must fail the run: timing two + // non-equivalent implementations produces a meaningless ratio, and a + // printed warning would still exit 0 and look like a good measurement. + long a = inline(), b = hoisted(); + if (a != b) + throw new InvalidOperationException($"{shape} n={n}: MISMATCH {a} vs {b}"); + + double ti = Bench(inline, reps), th = Bench(hoisted, reps); + Console.WriteLine($"{shape,-38} n={n,-7} inline={ti,9:F4} ms hoisted={th,9:F4} ms penalty = {ti / th,8:F1}x"); + } + + static void Main() + { + const int Iters = 1000; + Console.WriteLine("Same syntactic shape everywhere: a loop-invariant call in a loop body."); + Console.WriteLine($"Inner loop = {Iters} iterations. 'penalty' = how much the un-hoisted version costs.\n"); + + foreach (int n in new[] { 4, 64, 4096, 100_000 }) + { + var list = Enumerable.Range(0, n).ToList(); + IEnumerable seq = list.Select(x => x); // hides ICollection fast path + int th = -1; // Any() hits on the FIRST element + + Row("A: Any(x => x > t) [hit@0]", n, () => A_Inline(list, th, Iters), () => A_Hoisted(list, th, Iters), 20); + Row("A: Any(x => x > t) [miss]", n, () => A_Inline(list, int.MaxValue, Iters), () => A_Hoisted(list, int.MaxValue, Iters), 5); + Row("B: Count() on Select-wrapped IEnumerable", n, () => B_Inline(seq, Iters), () => B_Hoisted(seq, Iters), 5); + Row("C: OrderBy().First()", n, () => C_Inline(list, Iters), () => C_Hoisted(list, Iters), n > 10000 ? 1 : 3); + Row("D: .Count property (cheap)", n, () => D_Inline(list, Iters), () => D_Hoisted(list, Iters), 50); + Console.WriteLine(); + } + Console.WriteLine($"(sink={sink})"); + } +} diff --git a/docs/notes/invariant-cost-data/linq.csproj b/docs/notes/invariant-cost-data/linq.csproj new file mode 100644 index 00000000..2d643a65 --- /dev/null +++ b/docs/notes/invariant-cost-data/linq.csproj @@ -0,0 +1,8 @@ + + + Exe + net9.0 + disable + true + + diff --git a/docs/notes/invariant-cost-data/run.sh b/docs/notes/invariant-cost-data/run.sh new file mode 100755 index 00000000..34e5fcea --- /dev/null +++ b/docs/notes/invariant-cost-data/run.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# Reproduces the table in ../invariant-cost-static-vs-runtime.md. +# Requires a .NET 9 SDK. The published numbers were taken on 9.0.316; any 9.0.x +# should reproduce the ORDER and the spread, but exact multipliers will differ +# by machine and patch level -- the selected SDK/runtime is printed below so a +# rerun is self-describing rather than merely claiming reproduction. +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Only clean up a workdir we created ourselves; never delete a caller's. +if [[ $# -ge 1 ]]; then + WORK="$1" +else + WORK="$(mktemp -d)" + trap 'rm -rf -- "$WORK"' EXIT +fi + +mkdir -p "$WORK"; cp "$HERE"/{Program.cs,linq.csproj} "$WORK/" +cd "$WORK" + +echo "== toolchain actually used ==" +dotnet --version +dotnet --list-runtimes | grep -E '^Microsoft\.NETCore\.App 9\.0\.' || true +echo + +dotnet build -c Release -v q --nologo +# Tiering disabled: every method is compiled straight to FullOpts, so the +# measurement is steady-state code. (This is non-tiered mode -- "tier 1" is a +# tiering concept and does not apply when tiering is off.) +DOTNET_TieredCompilation=0 dotnet bin/Release/net9.0/linq.dll diff --git a/docs/notes/invariant-cost-static-vs-runtime.md b/docs/notes/invariant-cost-static-vs-runtime.md new file mode 100644 index 00000000..bdc19de1 --- /dev/null +++ b/docs/notes/invariant-cost-static-vs-runtime.md @@ -0,0 +1,141 @@ +# A loop-invariant call costs between 1× and 1092× — and only runtime knows which + +Working note. **Trigger:** the follow-up to +[`llvm-codegen-feasibility.md`](llvm-codegen-feasibility.md). That note closed +with "loop-invariant expensive query inside a loop is a static-analysis target, +not a codegen target". The objection that followed is correct and is the reason +this note exists: + +> Statically we can reach the *shape*. But we will not know what actually +> happens at runtime — and that is the killer feature. RyuJIT may handle it +> fine, or it may not, as in the case where it was crawling. Static analysis +> says "this could be suboptimal"; only measurement says **how bad**. + +This note measures the spread, and it is wider than the objection assumed. +Harness in [`invariant-cost-data/`](invariant-cost-data/), `run.sh` reproduces. + +## Method + +Four call shapes, each written twice — evaluated inside the loop vs hoisted by +hand — over collection sizes 4 … 100 000, inner loop fixed at 1000 iterations. +`penalty = inline / hoisted`. The harness asserts both variants return the same +value and **throws** if they do not, so a non-equivalent pair cannot be reported +as a measurement. .NET 9.0.316, `DOTNET_TieredCompilation=0` — tiering off, so +every method is compiled straight to FullOpts (non-tiered mode; "tier 1" is a +tiering concept that does not apply when tiering is disabled). + +**Every row is the same syntactic shape**: a loop-invariant call in a loop body. +A static rule matching that shape sees all of them as one finding. + +## Result + +| shape | n=4 | n=64 | n=4 096 | n=100 000 | +|---|---|---|---|---| +| `A: Any(x => x > t)` — predicate hits at element 0 | 2.8× | 12.6× | 11.1× | **8.3×** | +| `A: Any(x => x > t)` — predicate never hits | 12.2× | 164× | 898× | **1025×** | +| `B: Count()` on a `Select`-wrapped sequence | 53× | 347× | 936× | 1025× | +| `C: OrderBy().First()` | 362× | 822× | 1092× | 1003× | +| `D: .Count` property (O(1)) | 1.0× | 1.0× | 1.0×¹ | 1.0× | + +¹ one run showed 2.5× on sub-microsecond timings; that is measurement noise, not +an effect. + +## What the numbers actually say + +**1. The dynamic range is three orders of magnitude — 1.0× to 1092×.** A static +rule that reports all of these identically is not wrong, but *it cannot rank +this sample without runtime data*. Row D matches the shape perfectly and shows +**no measurable penalty in this run** — `List.Count` is an O(1) property — so on +this evidence it is the false-positive shape. + +**2. The killer row is A, and it settles the argument.** Compare the two `A` +rows at n=100 000: **8.3×** versus **1025×**. The source code is +**character-for-character identical** — `data.Any(x => x > threshold)`. The only +difference is the runtime *value* of `threshold` and the data distribution: +when the predicate hits at element 0, `Any` short-circuits and collection size +becomes irrelevant; when it never hits, the call is O(n) and the cost tracks n. + +A **local** rule — one reasoning about the call site without runtime or +call-site values — cannot separate those two, because at that site the two +programs *are* the same program. The honest scope: in this harness `Main` builds +its data with a deterministic `Enumerable.Range` and passes constant thresholds, +so a whole-program analyzer could in principle constant-propagate these +particular call sites. What it could not do is predict the cost for **general, +unknown inputs** — a `threshold` from configuration, a collection from a +database — which is the case the rule would actually face. The cost lives in +data the analyzer does not have. + +**3. Collection size does not predict cost either.** Row A[hit] is *flat* across +n (2.8 → 12.6 → 11.1 → 8.3), while row A[miss] grows a hundredfold over the same +range. So even "flag it only for large collections" — the obvious heuristic +rescue — is wrong in both directions. + +## The consequence: this is Layer 1 → Layer 2, and we already have that shape + +The conclusion is not "the static rule is worthless". It is that **the static +rule and the profiler are each individually unactionable, and complete each +other exactly**: + +- **A profiler alone** tells you `data.Any(...)` is 30% of CPU. It does *not* + tell you the call is loop-invariant, i.e. that the fix is free and safe. Hot + is not the same as fixable, and most hot lines are hot because they do + necessary work. +- **A static rule alone** tells you the call is provably invariant and gives the + exact fix — hoist it. It cannot tell you whether that fix buys 0% or 99.9%, + so it cannot rank, and an unranked list at 1.0×-to-1092× precision is noise. +- **Together**: *"this line is 30% of CPU **and** it is provably loop-invariant + → hoist it, here is the edit."* That is a finding with a magnitude, a proof, + and a patch. Neither layer produces it alone. + +This is precisely the architecture [`Plan.md`](../../Plan.md) §1 already +describes — Layer 1 static → Layer 2 runtime → Layer 3 AI over finished +evidence — and precisely the confirmation pattern already used for subscription +leaks, where own-check flags and the runtime harness confirms retention +(`MED` static, `HIGH` once runtime-confirmed, Plan.md §2 category 2). The +performance case reuses the mechanism unchanged; only the witness differs +(a timer/profiler sample instead of a heap walk). + +It also **repairs the objection this note's parent raised against itself**. That +note called "expensive" the kind of word that generates false positives, and +that is right — *as a static predicate*. Measured, it stops being a predicate +and becomes a number. The runtime layer is what makes the static rule shippable, +not an optional enhancement to it. + +## Honest limits + +- **Microbenchmark, one machine.** These are isolated shapes with no cache + pressure from surrounding work; real applications will compress the extremes. + The *ordering* and the fact of the spread are the claim, not the exact + multipliers. +- **This is not evidence the rule is implementable.** Proving loop-invariance + for a LINQ chain requires proving the receiver, the captured locals, *and* the + lambda are all effect-free across the loop body — real interprocedural work, + not a syntactic match. [P-036](../proposals/P-036-interprocedural-semantic-architecture.md) + is the right *host* for it (call graph, `MethodSummary`, SCC composition), but + **none of its five summary domains** — ownership, obligation, progress, + region, task — is a purity/effect-freedom domain, so this would be a sixth + one, and effects are owned by [P-008](../proposals/P-008-effects-and-resources.md) + (draft, explicitly horizon). Nothing here estimates that cost. +- **P-036's own rules would refuse the claim by default.** Its unknown/external + call policy classifies `Any(userLambda)` as an unresolved or unsupported + target: conservative defaults, recorded degraded precision, and the standing + rule that a check "may not pretend the call was proven harmless". So the + static half would emit a *candidate with declared uncertainty*, not a verdict + — which is another way of arriving at the same conclusion as the body of this + note: the magnitude has to come from the runtime layer. +- **`.NET 4.7.2` was not measured, and these numbers are not comparable to it.** + The trigger case was on Framework, which has a different JIT and a different + LINQ implementation; no lower-bound (or upper-bound) inference about Framework + follows from a .NET 9 run. Claiming one would need a matched Framework + measurement, which this harness does not do. +- **No corpus.** Four shapes chosen to span the range. Whether real code + clusters near 1× or near 1000× is unmeasured, and that distribution is what + would actually decide whether the rule is worth building. + +## Status + +**Recorded, not scheduled.** Notes record, the ROADMAP schedules. If this is +ever picked up, the honest first step is not the rule — it is +**mining (P-012) to find out whether the pattern occurs in real C# at all, and +with which cost distribution.** Building a checker for a pattern whose real-world +cost profile is unknown is how the 1.0× false positives get shipped. diff --git a/docs/notes/llvm-codegen-feasibility-data/Kernels.cs b/docs/notes/llvm-codegen-feasibility-data/Kernels.cs new file mode 100644 index 00000000..ee075ddb --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/Kernels.cs @@ -0,0 +1,172 @@ +using System; +using System.Diagnostics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; + +// Kernels 2-4: is the LLVM advantage general, or confined to straight-line +// numeric loops? Run with a long warmup so every method is at tier-1. +static class Kernels +{ + const int N = 1 << 16; + const int Reps = 4000; + const int Warm = 5000; + + [MethodImpl(MethodImplOptions.NoInlining)] + static long SumManaged(int[] a) + { + long s = 0; + for (int i = 0; i < a.Length; i++) s += a[i]; + return s; + } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long FilterSumManaged(int[] a, int lo, int hi) + { + long s = 0; + for (int i = 0; i < a.Length; i++) + { + int v = a[i]; + if (v > lo && v < hi) s += v * 2; else s -= v; + } + return s; + } + + // K3 control A: hand-written branchless C# (encourage cmov, kill the + // misprediction cost without any SIMD). + [MethodImpl(MethodImplOptions.NoInlining)] + static long FilterSumBranchless(int[] a, int lo, int hi) + { + long s = 0; + for (int i = 0; i < a.Length; i++) + { + int v = a[i]; + int inRange = (v > lo & v < hi) ? 1 : 0; + s += inRange * (v * 2) + (1 - inRange) * (-v); + } + return s; + } + + // K3 control B: hand-written SIMD with a select -- the same transform + // LLVM's vectorizer applies automatically (if-conversion + vectorize). + [MethodImpl(MethodImplOptions.NoInlining)] + static long FilterSumSimd(int[] a, int lo, int hi) + { + int w = System.Runtime.Intrinsics.Vector256.Count, i = 0; + var vlo = System.Runtime.Intrinsics.Vector256.Create(lo); + var vhi = System.Runtime.Intrinsics.Vector256.Create(hi); + // Accumulate in 64-bit to match the scalar version's `long s` exactly: + // widen each 8x32 result to 2x 4x64 before adding. + var accLo = System.Runtime.Intrinsics.Vector256.Zero; + var accHi = System.Runtime.Intrinsics.Vector256.Zero; + for (; i <= a.Length - w; i += w) + { + var v = System.Runtime.Intrinsics.Vector256.LoadUnsafe(ref a[i]); + var mask = System.Runtime.Intrinsics.Vector256.GreaterThan(v, vlo) + & System.Runtime.Intrinsics.Vector256.LessThan(v, vhi); + var sel = System.Runtime.Intrinsics.Vector256.ConditionalSelect(mask, v + v, -v); + (var wLo, var wHi) = System.Runtime.Intrinsics.Vector256.Widen(sel); + accLo += wLo; accHi += wHi; + } + long s = System.Runtime.Intrinsics.Vector256.Sum(accLo) + + System.Runtime.Intrinsics.Vector256.Sum(accHi); + for (; i < a.Length; i++) + { + int v = a[i]; + if (v > lo && v < hi) s += v * 2; else s -= v; + } + return s; + } + + sealed class Node { public Node Next; public long Value; } + + [MethodImpl(MethodImplOptions.NoInlining)] + static long ChaseManaged(Node head, long n) + { + long s = 0; + for (long i = 0; i < n && head != null; i++) { s += head.Value; head = head.Next; } + return s; + } + + [DllImport("native2", EntryPoint = "sum_native")] + static extern unsafe long SumNative(int* a, long n, long la); + [DllImport("native2", EntryPoint = "filter_sum_native")] + static extern unsafe long FilterSumNative(int* a, long n, long la, int lo, int hi); + // check-free variants: the frontend hoisted the range check out of the loop + [DllImport("native3", EntryPoint = "sum_free")] + static extern unsafe long SumFree(int* a, long n, long la); + [DllImport("native3", EntryPoint = "filter_sum_free")] + static extern unsafe long FilterSumFree(int* a, long n, long la, int lo, int hi); + + static double Time(Action f, int reps) + { + var sw = Stopwatch.StartNew(); + for (int r = 0; r < reps; r++) f(); + sw.Stop(); + return sw.Elapsed.TotalMilliseconds; + } + + public static unsafe void Run() + { + var a = new int[N]; + var rnd = new Random(42); + for (int i = 0; i < N; i++) a[i] = rnd.Next(0, 1000); + + long sink = 0; + for (int r = 0; r < Warm; r++) sink += SumManaged(a) + FilterSumManaged(a, 200, 800) + + FilterSumBranchless(a, 200, 800) + FilterSumSimd(a, 200, 800); + fixed (int* pa = a) + { + for (int r = 0; r < Warm; r++) sink += SumNative(pa, N, N) + FilterSumNative(pa, N, N, 200, 800) + + SumFree(pa, N, N) + FilterSumFree(pa, N, N, 200, 800); + + int* p = pa; + double t; + double elems = (double)N * Reps; + + Console.WriteLine($"\n-- K2 reduction (sum) --"); + t = Time(() => sink += SumManaged(a), Reps); + double k2m = elems / t / 1e6; + Console.WriteLine($"RyuJIT managed : {t,8:F1} ms {k2m,6:F2} Gelem/s 1.00x"); + t = Time(() => sink += SumNative(p, N, N), Reps); + Console.WriteLine($"LLVM -O3 w/ check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k2m,5:F2}x"); + t = Time(() => sink += SumFree(p, N, N), Reps); + Console.WriteLine($"LLVM -O3 no check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k2m,5:F2}x"); + + // correctness gate: every K3 variant must return the SAME value, and + // that value must be the pinned expected result -- comparing the + // variants only to each other would let a shared defect pass. + const long ExpectedFilterSum = 25_830_282; + long r0 = FilterSumManaged(a, 200, 800), r1 = FilterSumBranchless(a, 200, 800), + r2 = FilterSumSimd(a, 200, 800), r3 = FilterSumNative(p, N, N, 200, 800); + long r4 = FilterSumFree(p, N, N, 200, 800); + if (r0 != ExpectedFilterSum || r0 != r1 || r0 != r2 || r0 != r3 || r0 != r4) + throw new Exception($"K3 MISMATCH expected={ExpectedFilterSum} scalar={r0} " + + $"branchless={r1} simd={r2} native={r3} free={r4}"); + long q0 = SumManaged(a), q1 = SumNative(p, N, N), q2 = SumFree(p, N, N); + if (q0 != q1 || q0 != q2) throw new Exception($"K2 MISMATCH {q0} vs {q1} vs {q2}"); + Console.WriteLine($"\n-- K3 data-dependent branch (filter+sum) -- [all variants agree: {r0}]"); + t = Time(() => sink += FilterSumManaged(a, 200, 800), Reps); + double k3m = elems / t / 1e6; + Console.WriteLine($"RyuJIT managed : {t,8:F1} ms {k3m,6:F2} Gelem/s 1.00x"); + t = Time(() => sink += FilterSumBranchless(a, 200, 800), Reps); + Console.WriteLine($"RyuJIT branchless: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k3m,5:F2}x"); + t = Time(() => sink += FilterSumSimd(a, 200, 800), Reps); + Console.WriteLine($"RyuJIT SIMD+sel : {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k3m,5:F2}x"); + t = Time(() => sink += FilterSumNative(p, N, N, 200, 800), Reps); + Console.WriteLine($"LLVM -O3 w/ check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k3m,5:F2}x"); + t = Time(() => sink += FilterSumFree(p, N, N, 200, 800), Reps); + Console.WriteLine($"LLVM -O3 no check: {t,8:F1} ms {elems / t / 1e6,6:F2} Gelem/s {(elems / t / 1e6) / k3m,5:F2}x"); + } + + // K4: managed pointer chasing has no native counterpart worth timing + // (the object graph lives in the GC heap); measured to show the shape + // of code where no vectorizer of any kind helps. + const int Len = 1 << 14; + Node head = null; + for (int i = 0; i < Len; i++) head = new Node { Next = head, Value = i }; + double tc = Time(() => sink += ChaseManaged(head, Len), Reps / 4); + Console.WriteLine($"\n-- K4 pointer chase (managed only) --"); + Console.WriteLine($"RyuJIT managed : {tc,8:F1} ms {(double)Len * (Reps / 4) / tc / 1e6,6:F2} Gnode/s"); + Console.WriteLine($"\n(sink={sink})"); + } +} diff --git a/docs/notes/llvm-codegen-feasibility-data/Program.cs b/docs/notes/llvm-codegen-feasibility-data/Program.cs new file mode 100644 index 00000000..45826e38 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/Program.cs @@ -0,0 +1,109 @@ +using System; +using System.Diagnostics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +static class Bench +{ + const int N = 1 << 16; // 64K ints per array = 256 KB, fits in L2 + const int Reps = 4000; + + [MethodImpl(MethodImplOptions.NoInlining)] + static void AxpyManaged(int[] dst, int[] a, int[] b) + { + for (int i = 0; i < dst.Length; i++) + dst[i] = a[i] + b[i] * 3; + } + + // Control: what a C# developer can already write today, no LLVM involved. + [MethodImpl(MethodImplOptions.NoInlining)] + static void AxpySimd(int[] dst, int[] a, int[] b) + { + int w = Vector256.Count, i = 0; + var three = Vector256.Create(3); + for (; i <= dst.Length - w; i += w) + { + var va = Vector256.LoadUnsafe(ref a[i]); + var vb = Vector256.LoadUnsafe(ref b[i]); + (va + vb * three).StoreUnsafe(ref dst[i]); + } + for (; i < dst.Length; i++) dst[i] = a[i] + b[i] * 3; + } + + // Control 2: hand-SIMD + 4x unroll, i.e. exactly what LLVM's vectorizer + // chose on its own (width 8, interleave 4). Written by hand in C#. + [MethodImpl(MethodImplOptions.NoInlining)] + static void AxpySimdUnrolled(int[] dst, int[] a, int[] b) + { + int w = Vector256.Count, i = 0; + var three = Vector256.Create(3); + for (; i <= dst.Length - 4 * w; i += 4 * w) + { + (Vector256.LoadUnsafe(ref a[i]) + Vector256.LoadUnsafe(ref b[i]) * three).StoreUnsafe(ref dst[i]); + (Vector256.LoadUnsafe(ref a[i + w]) + Vector256.LoadUnsafe(ref b[i + w]) * three).StoreUnsafe(ref dst[i + w]); + (Vector256.LoadUnsafe(ref a[i + 2 * w]) + Vector256.LoadUnsafe(ref b[i + 2 * w]) * three).StoreUnsafe(ref dst[i + 2 * w]); + (Vector256.LoadUnsafe(ref a[i + 3 * w]) + Vector256.LoadUnsafe(ref b[i + 3 * w]) * three).StoreUnsafe(ref dst[i + 3 * w]); + } + for (; i < dst.Length; i++) dst[i] = a[i] + b[i] * 3; + } + + [DllImport("native", EntryPoint = "axpy_noalias")] + static extern unsafe void AxpyNoalias(int* dst, int* a, int* b, long n, long ld, long la, long lb); + + [DllImport("native", EntryPoint = "axpy_alias")] + static extern unsafe void AxpyAlias(int* dst, int* a, int* b, long n, long ld, long la, long lb); + + static unsafe void Main() + { + var dst = new int[N]; var a = new int[N]; var b = new int[N]; + for (int i = 0; i < N; i++) { a[i] = i; b[i] = N - i; } + + // Warm up well past the tier-1 / OSR thresholds. 200 reps is NOT + // enough -- it leaves tier-0 code in the measurement (see the note). + for (int r = 0; r < 5000; r++) AxpyManaged(dst, a, b); + for (int r = 0; r < 5000; r++) AxpySimd(dst, a, b); + for (int r = 0; r < 5000; r++) AxpySimdUnrolled(dst, a, b); + fixed (int* pd = dst, pa = a, pb = b) + { + for (int r = 0; r < 5000; r++) AxpyNoalias(pd, pa, pb, N, N, N, N); + for (int r = 0; r < 5000; r++) AxpyAlias(pd, pa, pb, N, N, N, N); + } + + long checksum = 0; + var sw = new Stopwatch(); + + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpyManaged(dst, a, b); + sw.Stop(); var tManaged = sw.Elapsed.TotalMilliseconds; checksum += dst[N - 1]; + + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpySimd(dst, a, b); + sw.Stop(); var tSimd = sw.Elapsed.TotalMilliseconds; checksum += dst[N - 1]; + + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpySimdUnrolled(dst, a, b); + sw.Stop(); var tSimdU = sw.Elapsed.TotalMilliseconds; checksum += dst[N - 1]; + + fixed (int* pd = dst, pa = a, pb = b) + { + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpyNoalias(pd, pa, pb, N, N, N, N); + sw.Stop(); var tNoalias = sw.Elapsed.TotalMilliseconds; checksum += dst[N - 1]; + + sw.Restart(); + for (int r = 0; r < Reps; r++) AxpyAlias(pd, pa, pb, N, N, N, N); + sw.Stop(); var tAlias = sw.Elapsed.TotalMilliseconds; + + double elems = (double)N * Reps; + Console.WriteLine($"N={N} reps={Reps} checksum={checksum}"); + Console.WriteLine($"RyuJIT managed : {tManaged,8:F1} ms {elems / tManaged / 1e6,6:F2} Gelem/s 1.00x"); + Console.WriteLine($"RyuJIT Vector256 : {tSimd,8:F1} ms {elems / tSimd / 1e6,6:F2} Gelem/s {tManaged / tSimd,5:F2}x"); + Console.WriteLine($"RyuJIT Vec256 x4 : {tSimdU,8:F1} ms {elems / tSimdU / 1e6,6:F2} Gelem/s {tManaged / tSimdU,5:F2}x"); + Console.WriteLine($"LLVM -O3 noalias : {tNoalias,8:F1} ms {elems / tNoalias / 1e6,6:F2} Gelem/s {tManaged / tNoalias,5:F2}x"); + Console.WriteLine($"LLVM -O3 alias : {tAlias,8:F1} ms {elems / tAlias / 1e6,6:F2} Gelem/s {tManaged / tAlias,5:F2}x"); + } + + Kernels.Run(); + } +} diff --git a/docs/notes/llvm-codegen-feasibility-data/README.md b/docs/notes/llvm-codegen-feasibility-data/README.md new file mode 100644 index 00000000..0a110f7a --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/README.md @@ -0,0 +1,30 @@ +# Harness for `../llvm-codegen-feasibility.md` + +Reproduces every number in the note. Requires `clang` (tested 18.1.3), the +.NET SDK 9 (tested 9.0.316), and an x86-64-v3 CPU. + +```console +$ ./run.sh [workdir] +``` + +| File | Role | +|---|---| +| `lenprobe.c` | 2x2: array length re-loaded vs hoisted, x aliasing unknown vs `restrict` | +| `matrix.c` | 2x2: aliasing x range-check-eliminated, lengths passed as arguments | +| `checkfree.c` | the K2/K3 kernels with the range check hoisted out of the loop | +| `native.c` | K1 (`dst[i] = a[i] + b[i]*3`), noalias and aliasing variants | +| `native2.c` | K2 reduction, K3 data-dependent branch, K4 pointer chase | +| `Program.cs` | K1 harness: RyuJIT scalar / `Vector256` / `Vector256` x4 / native | +| `Kernels.cs` | K2-K4 harness, incl. the correctness gate across all five K3 variants | + +Two things the harness enforces on purpose, both because they were got wrong +first (see "Two measurement traps" in the note): + +- **`DOTNET_TieredCompilation=0` plus a 5000-call warmup.** Tiering off means + everything compiles straight to FullOpts (non-tiered mode). A short warmup + under the tiered default measures partly-tier-0 code and inflates LLVM's + advantage by ~40%. +- **A correctness gate.** All **five** K3 variants must agree *and* equal the + pinned constant `25830282` (comparing them only to each other would let a + shared defect through); an unequal + accumulator width silently makes one variant do less work. diff --git a/docs/notes/llvm-codegen-feasibility-data/bench.csproj b/docs/notes/llvm-codegen-feasibility-data/bench.csproj new file mode 100644 index 00000000..c0f7e09f --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/bench.csproj @@ -0,0 +1,10 @@ + + + Exe + net9.0 + disable + true + true + false + + diff --git a/docs/notes/llvm-codegen-feasibility-data/checkfree.c b/docs/notes/llvm-codegen-feasibility-data/checkfree.c new file mode 100644 index 00000000..f0a6fe92 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/checkfree.c @@ -0,0 +1,13 @@ +#include +__attribute__((noreturn)) static void own_throw_ioor(void) { abort(); } +// same kernels, but the range check is hoisted out of the loop (frontend RCE) +long sum_free(const int *restrict a, long n, long la) { + if (n > la) own_throw_ioor(); + long s = 0; for (long i = 0; i < n; i++) s += a[i]; return s; +} +long filter_sum_free(const int *restrict a, long n, long la, int lo, int hi) { + if (n > la) own_throw_ioor(); + long s = 0; + for (long i = 0; i < n; i++) { int v = a[i]; if (v > lo && v < hi) s += v*2; else s -= v; } + return s; +} diff --git a/docs/notes/llvm-codegen-feasibility-data/lenprobe.c b/docs/notes/llvm-codegen-feasibility-data/lenprobe.c new file mode 100644 index 00000000..470ae98c --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/lenprobe.c @@ -0,0 +1,49 @@ +// Isolate the real blocker: is it aliasing, the range check, or the fact that +// `ldlen` is a MEMORY LOAD inside the loop (array length not known immutable)? +#include +void own_throw_ioor(void) __attribute__((noreturn)); +typedef struct { long len; int data[]; } Arr; // a .NET array object + +// 1. length reloaded from the object header every iteration, no alias info. +// This is what a naive CIL->LLVM frontend emits for ldlen. +void p1_reload_alias(Arr *dst, Arr *a, Arr *b, long n) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)a->len) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)b->len) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)dst->len) own_throw_ioor(); + dst->data[i] = a->data[i] + b->data[i] * 3; + } +} + +// 2. same, but the three objects are known distinct (type-safety / ownership). +void p2_reload_noalias(Arr *restrict dst, Arr *restrict a, Arr *restrict b, long n) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)a->len) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)b->len) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)dst->len) own_throw_ioor(); + dst->data[i] = a->data[i] + b->data[i] * 3; + } +} + +// 3. lengths hoisted once (frontend knows array length is IMMUTABLE), aliasing +// still unknown, per-element range check kept. +void p3_hoistlen_alias(Arr *dst, Arr *a, Arr *b, long n) { + long la = a->len, lb = b->len, ld = dst->len; + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst->data[i] = a->data[i] + b->data[i] * 3; + } +} + +// 4. lengths hoisted + noalias. +void p4_hoistlen_noalias(Arr *restrict dst, Arr *restrict a, Arr *restrict b, long n) { + long la = a->len, lb = b->len, ld = dst->len; + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst->data[i] = a->data[i] + b->data[i] * 3; + } +} diff --git a/docs/notes/llvm-codegen-feasibility-data/matrix.c b/docs/notes/llvm-codegen-feasibility-data/matrix.c new file mode 100644 index 00000000..13825850 --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/matrix.c @@ -0,0 +1,37 @@ +// 2x2: {alias info} x {range checks eliminated} -> does LLVM -O3 vectorize? +#include +void own_throw_ioor(void) __attribute__((noreturn)); + +// --- A: aliasing unknown, per-element range check (naive CIL lowering) --- +void a_alias_check(int *dst, const int *a, const int *b, long n, long ld, long la, long lb) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst[i] = a[i] + b[i] * 3; + } +} + +// --- B: noalias, per-element range check --- +void b_noalias_check(int *restrict dst, const int *restrict a, const int *restrict b, + long n, long ld, long la, long lb) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst[i] = a[i] + b[i] * 3; + } +} + +// --- C: aliasing unknown, checks hoisted out of the loop (frontend RCE) --- +void c_alias_nocheck(int *dst, const int *a, const int *b, long n, long ld, long la, long lb) { + if (n > la || n > lb || n > ld) own_throw_ioor(); + for (long i = 0; i < n; i++) dst[i] = a[i] + b[i] * 3; +} + +// --- D: noalias + checks hoisted --- +void d_noalias_nocheck(int *restrict dst, const int *restrict a, const int *restrict b, + long n, long ld, long la, long lb) { + if (n > la || n > lb || n > ld) own_throw_ioor(); + for (long i = 0; i < n; i++) dst[i] = a[i] + b[i] * 3; +} diff --git a/docs/notes/llvm-codegen-feasibility-data/native.c b/docs/notes/llvm-codegen-feasibility-data/native.c new file mode 100644 index 00000000..c75a4e2e --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/native.c @@ -0,0 +1,29 @@ +// Native side of the RyuJIT-vs-LLVM comparison. Same kernel, same semantics +// (per-element range check that traps), compiled by clang -O3. +#include +#include + +__attribute__((noreturn)) static void own_throw_ioor(void) { abort(); } + +// noalias + lengths hoisted (the "frontend knows array length is immutable" +// variant that LLVM vectorizes). +void axpy_noalias(int *restrict dst, const int *restrict a, const int *restrict b, + long n, long ld, long la, long lb) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst[i] = a[i] + b[i] * 3; + } +} + +// same, but aliasing unknown -> LLVM emits a runtime alias check + versioned loop. +void axpy_alias(int *dst, const int *a, const int *b, + long n, long ld, long la, long lb) { + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)lb) own_throw_ioor(); + if ((unsigned long)i >= (unsigned long)ld) own_throw_ioor(); + dst[i] = a[i] + b[i] * 3; + } +} diff --git a/docs/notes/llvm-codegen-feasibility-data/native2.c b/docs/notes/llvm-codegen-feasibility-data/native2.c new file mode 100644 index 00000000..7315d91d --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/native2.c @@ -0,0 +1,31 @@ +#include +__attribute__((noreturn)) static void own_throw_ioor(void) { abort(); } + +// K2: reduction. +long sum_native(const int *restrict a, long n, long la) { + long s = 0; + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + s += a[i]; + } + return s; +} + +// K3: data-dependent branch -- shaped like business logic, not a math kernel. +long filter_sum_native(const int *restrict a, long n, long la, int lo, int hi) { + long s = 0; + for (long i = 0; i < n; i++) { + if ((unsigned long)i >= (unsigned long)la) own_throw_ioor(); + int v = a[i]; + if (v > lo && v < hi) s += v * 2; else s -= v; + } + return s; +} + +// K4: pointer-chasing / indirection -- the shape of object-graph traversal. +typedef struct Node { struct Node *next; long value; } Node; +long chase_native(Node *head, long n) { + long s = 0; + for (long i = 0; i < n && head; i++) { s += head->value; head = head->next; } + return s; +} diff --git a/docs/notes/llvm-codegen-feasibility-data/run.sh b/docs/notes/llvm-codegen-feasibility-data/run.sh new file mode 100755 index 00000000..7d90e32a --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility-data/run.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# Reproduce the measurements in ../llvm-codegen-feasibility.md. +# +# Requires: clang (tested: 18.1.3), .NET SDK 9 (tested: 9.0.316), x86-64-v3 CPU. +# Usage: ./run.sh [workdir] +set -euo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +WORK="${1:-$(mktemp -d)}" +mkdir -p "$WORK/cs" + +echo "== part 1: which loop shapes does LLVM -O3 vectorize? ==" +for f in lenprobe matrix; do + echo "--- $f.c ---" + clang -O3 -march=x86-64-v3 -c "$HERE/$f.c" -o /dev/null \ + -Rpass=loop-vectorize -Rpass-missed=loop-vectorize 2>&1 | grep remark || true +done +echo "--- checkfree.c (range check hoisted out of the loop) ---" +clang -O3 -march=x86-64-v3 -c "$HERE/checkfree.c" -o /dev/null \ + -Rpass=loop-vectorize -Rpass-missed=loop-vectorize 2>&1 | grep remark || true + +echo +echo "== part 2: RyuJIT vs LLVM -O3, same kernels ==" +cp "$HERE"/{Program.cs,Kernels.cs,bench.csproj} "$WORK/cs/" +clang -O3 -march=x86-64-v3 -shared -fPIC "$HERE/native.c" -o "$WORK/cs/libnative.so" +clang -O3 -march=x86-64-v3 -shared -fPIC "$HERE/native2.c" -o "$WORK/cs/libnative2.so" +# native3 = native2 kernels + the check-hoisted variants from checkfree.c +cat "$HERE/native2.c" > "$WORK/native3.c" +sed '/^#include/d; /own_throw_ioor(void) { abort/d' "$HERE/checkfree.c" >> "$WORK/native3.c" +clang -O3 -march=x86-64-v3 -shared -fPIC "$WORK/native3.c" -o "$WORK/cs/libnative3.so" + +cd "$WORK/cs" +dotnet build -c Release -v q --nologo +cp ./*.so bin/Release/net9.0/ + +# TieredCompilation=0 measures steady-state code. With tiering on, the default +# warmup is NOT enough and the managed numbers come out ~1.4x pessimistic -- +# see the "Two measurement traps" section of the note. +for run in 1 2 3; do + DOTNET_TieredCompilation=0 dotnet bin/Release/net9.0/bench.dll + echo +done + +echo "== part 3: RyuJIT disassembly of the scalar loop ==" +DOTNET_JitDisasm="AxpyManaged" DOTNET_TieredCompilation=0 \ + dotnet bin/Release/net9.0/bench.dll 2>&1 | sed -n '/Assembly listing/,/^$/p' | head -60 diff --git a/docs/notes/llvm-codegen-feasibility.md b/docs/notes/llvm-codegen-feasibility.md new file mode 100644 index 00000000..18f01dbb --- /dev/null +++ b/docs/notes/llvm-codegen-feasibility.md @@ -0,0 +1,312 @@ +# Can .NET code get C/C++-grade compiler optimizations? — a measured answer + +Working note. **Trigger:** a design discussion asked whether .NET code could be +pushed through some representation where LLVM's "insane" C/C++ optimizations +apply, and whether a proof-of-concept is doable on a shoestring. A second +question rode along: *what does .NET actually optimize in our code?* — prompted +by an expensive LINQ query that visibly re-ran every iteration of a `for` loop +on .NET Framework 4.7.2. + +This note answers both **with measurements, not opinion**. The harness is +committed in [`llvm-codegen-feasibility-data/`](llvm-codegen-feasibility-data/) +and reproduced by `run.sh`, so every number below is falsifiable. + +**Bottom line up front:** + +1. **PoC is doable** — the pipeline is alive and the measurement ladder below + already runs. But it is a *codegen* project, not an Own.NET project. +2. **The headline result is negative for us.** The "ownership → `noalias` → + speed" story that makes Rust fast **did not reproduce**: alias metadata + bought **0%** on every kernel measured. The thing that unlocks LLVM is + **range-check elimination in the frontend**, which is ordinary compiler work + with no ownership content. +3. **LLVM is not a capability ceiling C# cannot reach.** Hand-written C# + `Vector256` code matched LLVM `-O3` on one kernel and **beat clang `-O3` by + 1.8×** on another. What LLVM sells is *automation*, not headroom. +4. RyuJIT is **already better than the folklore** — it hoists, clones loops, and + eliminates bounds checks. What it does not do is **vectorize or unroll**. + +## Method + +- **Environment.** Intel Xeon @ 2.10GHz, 4 vCPU, AVX-512 (`avx512f/bw/dq/vl/cd/ + ifma/vbmi`), Linux 6.18.5. clang/LLVM **18.1.3**, .NET SDK **9.0.316** + (runtime 9.0.18), `linux-x64`. Native side always + `clang -O3 -march=x86-64-v3`. +- **Shape of the comparison.** The same kernel is written twice — once in C# + (run by RyuJIT) and once in C (compiled by clang `-O3`, called via + `DllImport` over `fixed` pointers). The C is deliberately written to mimic + *naive CIL lowering*: an explicit per-element range check that calls a + `noreturn` throw helper, and (where noted) the array length re-loaded from + the object header rather than hoisted. +- **Controls are the point.** Every kernel carries a hand-written C# SIMD + variant, because "LLVM beats scalar C#" is a boring claim; "LLVM beats + *well-written* C#" is the claim that would justify building anything. +- **Correctness gate.** All **five** K3 variants — scalar, branchless, + hand-SIMD, native with the per-element check, and native with the check + hoisted — must return the identical value, and that value must equal the + pinned constant `25830282`, or the harness throws. An early version of the + SIMD control accumulated in 32-bit while the scalar accumulated in 64-bit — + it produced a flattering **33×** that was simply *less work*. The gate exists + because that mistake was made here. +- **Steady state.** Reported numbers disable tiered compilation + (`DOTNET_TieredCompilation=0`, i.e. everything is compiled straight to + FullOpts — not "tier 1", which is a tiering concept) and additionally warm up + 5000 calls so the tiered default is also at steady state. See the measurement + traps below — this is not a detail. + +### Two measurement traps (both hit during this work) + +- **Warmup.** With tiering on and a 200-call warmup, the managed kernels + measured **1.4× slower** than steady state — and that inflated LLVM's apparent + advantage from **3.3× to 4.6×**. The first draft of this note would have + overstated the result by 40%. 4000 *measured* iterations did not save it; + only a longer warmup or `TieredCompilation=0` did. +- **Unequal work.** See the correctness gate above. + +## Result 1 — what actually blocks LLVM on CIL-shaped code + +`lenprobe.c`, 2×2 over the same `dst[i] = a[i] + b[i]*3` loop +(`-Rpass=loop-vectorize`): + +| | length re-loaded each iteration (naive `ldlen`) | length hoisted once (immutability known) | +|---|---|---| +| **aliasing unknown** | ❌ not vectorized | ✅ vectorized (width 8, interleave 2) | +| **`noalias` (`restrict`)** | ❌ not vectorized | ✅ vectorized (width 8, interleave 2) | + +And `checkfree.c` vs `native2.c`, over the reduction and branchy kernels: + +| loop body | vectorized? | +|---|---| +| per-element range check + `noreturn` throw | ❌ **not vectorized** | +| range check hoisted out of the loop | ✅ vectorized (width 4, interleave 4) | + +**Reading.** A naive CIL→LLVM frontend emits, for every single `ldelem`, a +bounds check whose failure edge calls a `noreturn` throw helper. That extra loop +exit **switches LLVM's vectorizer off entirely** — and re-loading the array +length from the object header does the same, independently. `noalias` changes +**nothing** in either row. + +So the load-bearing contribution a CIL→LLVM frontend must make is *.NET-specific +invariants* — array-length immutability, and range-check elimination — **before** +LLVM ever sees the IR. LLVM will not recover them for you. This inverts the +intuition the idea started from: the win is not "hand CIL to LLVM and collect +C-grade optimizations", it is "do the .NET-specific proof work yourself, and +LLVM's vectorizer is the reward." + +## Result 2 — RyuJIT vs LLVM, measured + +Median of 3 runs, steady state. `Gelem/s` = elements processed per second. + +**K1 — `dst[i] = a[i] + b[i]*3` (straight-line numeric):** + +| variant | Gelem/s | ×scalar | +|---|---|---| +| RyuJIT scalar | 1.85 | 1.00× | +| RyuJIT `Vector256`, no unroll | 5.4 | 2.9× | +| **RyuJIT `Vector256` ×4 unrolled** | **6.4** | **3.5×** | +| LLVM `-O3`, `noalias` | 6.1 | 3.3× | +| LLVM `-O3`, aliasing unknown | 6.1 | 3.3× | + +**K2 — reduction (`s += a[i]`):** + +| variant | Gelem/s | ×scalar | +|---|---|---| +| RyuJIT scalar | 2.75 | 1.00× | +| LLVM `-O3`, per-element check | 2.75 | **1.00×** | +| LLVM `-O3`, check hoisted | 11.1 | 4.0× | + +**K3 — data-dependent branch (`if (v > lo && v < hi) s += v*2; else s -= v`), +random data, ~unpredictable branch — the shape of business logic:** + +| variant | Gelem/s | ×scalar | +|---|---|---| +| RyuJIT scalar | 0.22 | 1.00× | +| RyuJIT "branchless" (multiply trick) | 0.18 | **0.80×** | +| **RyuJIT hand-SIMD + `ConditionalSelect`** | **5.0** | **22.5×** | +| LLVM `-O3`, per-element check | 1.39 | 6.2× | +| LLVM `-O3`, check hoisted | 2.81 | 12.5× | + +**K4 — pointer chase over a managed object graph:** 0.55 Gnode/s. No vectorizer +of any kind applies; this is latency-bound and included to mark the boundary. + +### Reading the table + +- **`noalias` bought nothing.** K1: 6.1 vs 6.1. Over a 64K-element loop, LLVM's + runtime alias check amortizes to zero and the versioned loop runs at the same + speed as the `restrict` one. **The ownership→speed story does not reproduce + here.** This is the single most decision-relevant number in the note, and it + is the one that says *this is not an Own.NET project*. +- **LLVM did not vectorize K2/K3 at all** — inspection of the assembly shows 0 + `ymm`/`zmm` registers in `filter_sum_native` and 2 `cmov`s. Its entire 6.2× on + K3 is **if-conversion** (branch → `cmov`, killing mispredictions), not SIMD. + The bounds check blocked the vectorizer, exactly as Result 1 predicts. +- **Hand-written C# wins where it is written.** C# `Vector256` ×4 (6.4) edges + out LLVM on K1 (6.1), and hand-SIMD C# on K3 (5.0) **beats clang `-O3` (2.8) + by 1.8×** — because a human applied if-conversion *and* vectorization where + LLVM managed only the former. +- **Micro-optimizing by hand can backfire.** The "branchless" multiply trick + made K3 **20% slower** than the naive branch. RyuJIT did not turn it into a + `cmov`; it just did more arithmetic. + +## Result 3 — what .NET actually optimizes (the LINQ-in-a-loop question) + +RyuJIT disassembly of the scalar K1 loop (`DOTNET_JitDisasm`, FullOpts) — +abridged: + +```asm +G_M000_IG02: mov ecx, dword ptr [rdi+0x08] ; dst.Length -- hoisted, loaded ONCE +G_M000_IG03: test rsi, rsi ; null checks -- hoisted + cmp dword ptr [rsi+0x08], ecx ; a.Length >= dst.Length -- hoisted + cmp dword ptr [rdx+0x08], ecx ; b.Length >= dst.Length -- hoisted +G_M000_IG04: mov r9d, dword ptr [rsi+4*r8+0x10] + mov r10d, dword ptr [rdx+4*r8+0x10] + lea r10d, [r10+2*r10] ; *3 -> lea (strength reduction) + add r9d, r10d + mov dword ptr [rdi+4*r8+0x10], r9d + inc eax + cmp ecx, eax + jg SHORT G_M000_IG04 ; hot loop: ZERO bounds checks +``` + +That is **loop cloning**: RyuJIT proved the safe precondition once, then emits a +fast loop with **no bounds checks at all**, keeping a checked clone (`IG06`) as +fallback. Plus hoisting of lengths and null checks, and strength reduction +(`*3` → `lea`). RyuJIT is doing real optimization work. + +**What it does not do: vectorize or unroll.** That is the whole 3.3× gap on K1, +and auto-vectorization remains an open request upstream +([dotnet/runtime#11263](https://github.com/dotnet/runtime/issues/11263)). + +**So why was the LINQ query not hoisted out of that 4.7.2 loop?** Not a JIT bug, +and LLVM would not have fixed it either. A LINQ chain is a sequence of +**opaque, allocating, interface-dispatched calls**. Loop-invariant code motion +may only hoist an expression it can prove side-effect-free, and nothing in the +CLR gives the JIT purity information about `Where`/`Select`/`Any` over an +arbitrary user predicate. No compiler in this note's data — RyuJIT *or* clang +`-O3` — hoists an opaque call out of a loop; it is not a legal transform without +a purity proof. On 4.7.2 it is worse still: no dynamic PGO, no guarded +devirtualization, so the delegate and enumerator calls stay indirect. + +The actionable consequence is the interesting one: **this class of bug is a +static-analysis target, not a codegen target.** "Loop-invariant expensive query +evaluated inside a loop" is structurally visible in source — which is Own.NET's +existing business — whereas no amount of LLVM would touch it. If any part of +this discussion deserves to become a rule, it is that one, and it has nothing to +do with LLVM. + +## Landscape — who has already tried this + +| Effort | What it is | Status | +|---|---|---| +| **Unity Burst** | LLVM over a C# subset (HPC#), the real proof this works — auto-vectorization, `[NoAlias]`, 10–100× on compute kernels | **Alive, shipping.** Buys its wins by *restricting the language* (no GC, no classes, no exceptions in kernels) | +| **NativeAOT-LLVM** | `dotnet/runtimelab`, LLVM backend for NativeAOT, primarily WebAssembly | Experimental branch; LLVM also used as object writer in shipping NativeAOT | +| **LLILC** | Microsoft's LLVM-based JIT for CoreCLR (2015) | **Dead**, archived, superseded | +| **Mono LLVM backend** | `--llvm` AOT path | Long-standing, production for AOT targets | +| **`rustc_codegen_clr`** | the *reverse* direction (Rust → CIL) | Referenced for context only; solves a different problem | + +**Burst is the honest precedent, and it teaches the real lesson**: the way to +get LLVM-grade codegen out of C# is not to translate all of C# — it is to carve +out a **restricted subset** where the .NET-specific invariants are cheap to +establish. Which is precisely Result 1, arrived at independently. + +### What Burst actually is, and whether it helps us + +Burst compiles **HPC#** — "High-Performance C#", a deliberately crippled subset +— through LLVM instead of RyuJIT. The subset is the whole trick: no classes, no +managed heap references, no GC allocation, no exceptions in kernels; you work +over `struct`s and `NativeArray` inside Unity's job system, and you annotate +pointers `[NoAlias]`. Under those restrictions LLVM's auto-vectorizer, +unroller and inliner all apply, and Unity reports 10–100× on compute-heavy +kernels. + +Note *why* those restrictions matter, against Result 1: forbidding managed +references and using `NativeArray` removes the object header, so there is no +per-iteration `ldlen`; the job system's bounds are loop-invariant by +construction, so the range checks hoist. **Burst does not beat the blockers +found above — it defines them out of the language.** That is the entire +mechanism. + +Three honest conclusions about how it helps us, in decreasing order of value: + +1. **As a cost calibration — the main value.** Burst is what "success" costs. + A funded team, a decade, and a language subset severe enough that ordinary + C# does not compile. It converts "could we do this?" into "this is the size + of the thing", which is why rung 4+ is not recommended. +2. **As a design precedent, with a real but partial parallel here.** OwnLang is + already a restricted language with storage discipline enforced by + construction ([`spec/BufferPolicies.md`](../../spec/BufferPolicies.md): + `stack`/`scratch`/`pooled`/`native`, where B1 forbids stack buffers from + escaping). So the structural analogy `Burst : HPC#` ≈ + `LLVM backend : OwnSharp subset` is not fantasy — a language that already + pins storage and escape could supply the invariants a frontend needs. + **But** the parallel stops exactly where Result 2 does: the invariants worth + supplying are range-check elimination and length immutability, and the + ownership content contributes **0%**. OwnLang would be helping as *a + restricted language*, not as *an ownership system* — so this is not a reason + to build it, and it does not make the ownership work pay off in codegen. +3. **Not usable as a component.** Burst is Unity-coupled — the package, the job + system, `NativeArray`. There is nothing to vendor or reuse outside Unity. + +One transferable practice, independent of any of the above: Burst ships an +**inspector** showing the generated assembly per kernel, because even inside the +subset, whether a loop vectorised is not predictable from the source. That is +the same "measure, do not assume" discipline this note's `-Rpass` output and +`DOTNET_JitDisasm` dumps applied — and the same lesson as +[`invariant-cost-static-vs-runtime.md`](invariant-cost-static-vs-runtime.md). + +## Is a PoC doable? Yes — and here is the ladder + +Rungs 1–3 are *already done and committed here*; the remaining cost is rung 4+. + +1. ✅ **Does LLVM help CIL-shaped loops at all?** — `run.sh` part 1. Answered: + only after frontend RCE. +2. ✅ **How big is the prize vs RyuJIT?** — 3.3–4.0× on numeric kernels, 1.00× + when memory-bound, 0× on pointer chasing. +3. ✅ **Is the prize reachable without LLVM?** — yes, hand-SIMD C# matches or + beats it. This is the rung that decides the project. +4. ⬜ **A real IL→LLVM translator** for a tiny opcode subset (`ldarg`/`ldloc`/ + `ldc.i4`/arithmetic/`ldelem.i4`/`stelem.i4`/`ldlen`/branches/`ret`), locals + as `alloca` + `mem2reg`. Perhaps 300–500 lines. **Only worth writing after + rung 3 says the automation is worth having** — and here it says it mostly + is not. +5. ⬜ Everything real: GC write barriers, exact stack maps for a moving GC, + exception semantics, P/Invoke, generics/shared code, tiering and OSR. This + is where LLILC died, and it is a compiler-team-years problem, not a weekend. + +**Recommendation: do not build rung 4+ here.** Not because it fails — it works — +but because rung 3 shows the payoff is available today in plain C#, and Result 1 +shows the enabling work carries no ownership content, so it would not compound +with anything Own.NET does. The experiment was worth running precisely because +it produced a clean negative on the part that looked most attractive. + +## What this does and does not reopen + +- **Does not reopen the `tech-debt-register.md` rejection of MLIR/LLVM.** That + rejection is about *OwnIR as a fact-interchange format* (§2: "instruction-level + compiler frameworks, wrong abstraction"). This note asks a different question — + LLVM as a *codegen backend for execution* — and independently lands on "no", + for different reasons. Both stand; neither is evidence for the other. +- **Does not create a work item.** Recorded as research context, per the + `research-landscape-2026.md` discipline: notes record, ROADMAP schedules. +- **Does leave one genuinely actionable thread**, and it is not the LLVM one: + the loop-invariant-expensive-call rule from Result 3. Filed here as an + observation, not a proposal — it would need corpus evidence (P-012) before + anyone writes a rule, and "expensive" is exactly the kind of word that + generates false positives. + +## Honest limits of this measurement + +- **One machine, one microarchitecture, 4 vCPU shared.** One K1 run showed + ~30% variance from a noisy neighbour; the reported medians are stable across + 3 runs but this is not a clean benchmarking rig. +- **Four kernels is not a corpus.** They were chosen to span straight-line + numeric / reduction / branchy / pointer-chasing, but no claim is made that + they represent real application profiles. +- **The C side is a *proxy* for CIL lowering, not real CIL.** It was hand-written + to mimic what a naive frontend emits. Rung 4 would replace the proxy with an + actual translator; until then, Result 1 is a statement about *loop shapes*, + which is how it is phrased. +- **No GC interaction is modelled at all.** The native kernels operate on pinned + buffers. Write barriers and stack maps — the things that actually killed + LLILC — are entirely absent from these numbers, so nothing here should be read + as a cost estimate for a real backend.