diff --git a/README.md b/README.md index 0608c47..c4c6678 100644 --- a/README.md +++ b/README.md @@ -111,7 +111,7 @@ You will need the target's own build and test toolchain available on the runner. ### Steps -Fork this repository. Configure the run in [`.github/codeweave.config`](.github/codeweave.config), which holds the target repository URL and branch, the per-phase iteration caps, and the per-phase model schedules; none of this is hardcoded in the workflow logic. Replace the sample constraint in [`constraints/project.md`](constraints/project.md) with your target's real constraints, such as toolchain versions, build environment variables, and scope limits, and note that `constraints/harness.md` and `constraints/harness-context.md` are optional target-specific inputs for the harness-design phase. Add the two secrets. Then smoke-test the structure before spending model time: +Fork this repository. Configure the run in [`.github/codeweave.config`](.github/codeweave.config), which holds the target repository URL and branch, the per-phase iteration caps, and the per-phase model schedules; none of this is hardcoded in the workflow logic. Fill in the constraint template in [`constraints/project.md`](constraints/project.md) with your target's real constraints, such as toolchain versions, build environment variables, and scope limits, and note that `constraints/harness.md` and `constraints/harness-context.md` are optional target-specific templates for the harness-design phase. Add the two secrets. Then smoke-test the structure before spending model time: ```bash gh workflow run codeweave.yml -f dry_run=true @@ -237,7 +237,7 @@ Several limitations are worth stating plainly. The pipeline requires a self-host ## Documentation, community, and trust -The per-phase reference lives in [`docs/index.md`](docs/index.md), and the design rationale is in [`executive-summary.md`](executive-summary.md). Worked examples of the prompt and constraint files live under [`work/`](work) and [`constraints/`](constraints), and a sample run's artifacts appear in `proof/`, which is created automatically. `[ADD LINK TO A PUBLISHED EXAMPLE RUN]` +The per-phase reference lives in [`docs/index.md`](docs/index.md), and the design rationale is in [`executive-summary.md`](executive-summary.md). Worked examples of the prompt files live under [`work/`](work), the language-neutral constraint templates you fill in for your target live under [`constraints/`](constraints), and a sample run's artifacts appear in `proof/`, which is created automatically. `[ADD LINK TO A PUBLISHED EXAMPLE RUN]` The near-term direction is set out in [`ROADMAP.md`](ROADMAP.md), whose current focus is publishing a real run and verifying portability across additional targets through the manifest seam. Contributions are welcome; please read [`CONTRIBUTING.md`](CONTRIBUTING.md) first, and include the relevant `proof/` artifacts when you report pipeline behavior. For support, open a [GitHub issue](../../issues). `[ADD DISCUSSIONS LINK IF ENABLED]` Because the pipeline handles two access tokens and pushes branches to a target repository, please scope the tokens minimally as described under [Prerequisites](#quickstart) and report any vulnerability through [`SECURITY.md`](SECURITY.md). diff --git a/constraints/harness-context.md b/constraints/harness-context.md index ef0cf7a..391ec08 100644 --- a/constraints/harness-context.md +++ b/constraints/harness-context.md @@ -5,93 +5,88 @@ > This context supplements the book and ADRs — it does not override what the source code and ADRs actually show. > Delete or comment out any section that does not apply. +The sections below are prompts for the kind of domain context Phase 3 benefits from. They +are language and framework neutral. Fill each one in for your target, or remove it. + --- ## Codebase Purpose -PyTorch is a machine learning framework that provides tensor computation and automatic differentiation, primarily used for neural network training and inference. Its CPU inference path — tensor dispatch, operator kernels, autograd graph traversal, and BLAS-backed linear algebra — is the primary subject of this analysis. +Describe what the target does and which execution path is the subject of this analysis. Be +specific about the layer under investigation (for example a request-handling path, a +compute kernel, a parsing stage, or an inference loop) rather than the project as a whole. -The representative use case is text-generation inference: a transformer-based language model receives tokenised input, executes a forward pass through its layers, and produces output tokens in a sampling loop. This exercises PyTorch's dispatch stack, memory allocator, and linear algebra backends end-to-end on CPU. +Then describe the representative use case in one or two sentences: the concrete workload +that exercises that path end to end. Phase 3 turns this into the integration scenario. --- ## Observability Focus Areas -- **Operator dispatch**: routes Python-level tensor operations to C++ kernels — high call frequency, low latency budget; dominant source of per-token overhead for small tensors -- **Autograd engine**: builds and traverses the computation graph — relevant even in `torch.no_grad()` contexts due to graph teardown overhead -- **Memory allocator**: tensor allocation and deallocation patterns — allocation pressure increases with sequence length and batch size -- **BLAS/MKL-DNN threading**: controls parallelism for matrix multiplications — thread pool saturation or underutilisation shows up as CPU underutilisation on multi-core runners -- **Python/C++ boundary crossings**: the overhead of each sampling step includes Python dispatch overhead; repeated for every generated token +List the subsystems whose behaviour dominates the path above, and for each say why it +matters for performance or energy. Aim for the handful of areas a profiler would light up, +for example: + +- A high-frequency, low-latency-budget layer that dominates per-unit overhead. +- A memory allocation pattern whose pressure grows with input size. +- A parallelism or threading control whose saturation or underutilisation shows up as wasted CPU. +- A boundary crossing (process, language, or I/O) repeated on every unit of work. --- ## Representative Scenario Guidance -The integration scenario is a **CPU text-generation inference loop**. Phase 4 must generate this test such that all scenario parameters are loaded from configuration — not hardcoded. This preserves CodeWeave's generic character: swapping to a different model or prompt set requires only a configuration change, not a code change. +Describe the integration scenario Phase 4 should build. State clearly that **all scenario +parameters must be loaded from configuration, not hardcoded**, so swapping inputs requires +only a configuration change. This preserves CodeWeave's generic character. ### Configuration sources +List each scenario parameter, where it is read from, and its default: + | Parameter | Source | Default | |-----------|--------|---------| -| Model name | `GENAI_MODEL` environment variable | `distilgpt2` | -| Prompts | `integration-test/scenarios/prompts.json` (JSON array of strings) | 8 prompts below | -| Time limit (seconds) | `GENAI_MAX_SECONDS` environment variable | `30` | -| Generation seed | Hardcoded `torch.manual_seed(42)` | — | - -Phase 4 must generate the `integration-test/scenarios/prompts.json` file containing the default prompt set below. This file is the hand-off point between scenario configuration and test code. +| *example: workload size* | *environment variable* | *value* | +| *example: input set* | *a JSON file the phase generates* | *value* | +| *example: time limit* | *environment variable* | *value* | ### Hot loop structure +Sketch the measured loop in pseudocode so Phase 4 knows what to profile and what to keep +outside the profiled scope: + ``` -load model (from GENAI_MODEL) and tokenizer — outside profiled scope -start cProfile, torch.profiler, tracemalloc -while wall_clock < GENAI_MAX_SECONDS: - prompt = prompts[iteration % len(prompts)] - tokenize(prompt) → input_ids - model.generate(input_ids, max_new_tokens=60, do_sample=True, temperature=0.7, - top_k=50, repetition_penalty=1.3) - decode newly generated tokens only +set up inputs and dependencies — outside profiled scope +start profiler(s) and memory tracking +while wall_clock < time_limit: + pick the next input + run one unit of the workload under investigation + record the result accumulate iteration count and output log -stop cProfile, torch.profiler, tracemalloc -export Chrome trace (torch.profiler) -write cProfile stats +stop profiler(s) and memory tracking +export traces and profile stats ``` -Energy tracking is handled by the `conftest.py` CodeCarbon fixture (per benchmark-marked test invocation). - -### Default prompt set +State how energy tracking is wired in (for example a measurement fixture invoked per +benchmark-marked test). -Phase 4 writes the following to `integration-test/scenarios/prompts.json`: - -```json -[ - "Today a museum curator took the morning train from Amsterdam to Rotterdam. She carried a folder of restoration notes and a thermos of cold coffee.", - "Meanwhile a student with very little money was backpacking through Belgium, sleeping in hostels and eating bread from supermarket shelves.", - "A startup founder in Berlin was on her third cup of espresso, debugging a production incident that had started at 3 am.", - "In a small fishing village on the coast of Portugal, an elderly fisherman repaired his nets by hand while his grandson watched silently.", - "A software engineer in Tokyo was refactoring a legacy codebase, removing a comment that said 'fix this later' written six years ago.", - "A nurse finishing a night shift in a London hospital sat in the break room, staring at a lukewarm cup of tea, thinking about nothing in particular.", - "A journalist in Cairo was transcribing an interview, pausing every few seconds to replay a phrase she could not quite hear on the recording.", - "A retired teacher in rural France was writing a letter by hand to her former student, now living in Canada, about the summer storms that had flattened her garden." -] -``` +### Default input set -These prompts exercise: tokenization with varying lengths, multi-layer forward pass, autoregressive sampling, and output decoding. They are replaceable by editing `integration-test/scenarios/prompts.json` — no code change required. +If the scenario needs a fixed input set, describe it and note that Phase 4 writes it to a +configuration file that is the hand-off point between scenario configuration and test code. +State that the inputs are replaceable by editing that file, with no code change required. --- ## Known Performance Hotspots -- Kernel dispatch overhead for small tensors — every token step calls many aten ops with small shapes -- Memory allocation in the generation loop — a new tensor is allocated per output token -- Autograd graph overhead — `torch.no_grad()` suppresses gradient tracking but not all associated overhead -- BLAS thread pool utilisation — `torch.set_num_threads()` behaviour under sustained load affects throughput reproducibility +List any hotspots already suspected from the source, ADRs, or prior runs, with a one-line +reason for each. These seed the hotspot search; they do not constrain it. --- ## Out-of-Scope Subsystems -- CUDA/GPU backends: excluded by `constraints/project.md` (CPU-only execution) -- Distributed training (`torch.distributed`): out of scope for single-process inference analysis -- Model loading and tokenizer initialisation: excluded from the profiled hot loop (one-time setup cost, not inference bottleneck) -- Third-party vendored code under `third_party/` in the source tree: not owned by this project +List the subsystems explicitly excluded from analysis and why (for example excluded by a +scope constraint in `constraints/project.md`, one-time setup cost rather than steady-state +work, or third-party vendored code the project does not own). diff --git a/constraints/harness.md b/constraints/harness.md index ae7bfca..dec7926 100644 --- a/constraints/harness.md +++ b/constraints/harness.md @@ -5,71 +5,74 @@ > Phase 3 reads this file and incorporates its constraints into `integration-test/SOURCE-UNDER-INVESTIGATION.md`. > Delete or comment out any section that does not apply to your codebase. +The sections below describe the *kinds* of measurement constraint the harness needs. They +are language and framework neutral. Replace every placeholder with your target's real +values, and remove anything that does not apply. + --- ## Execution Environment -- CPU only — no GPU required; exclude all CUDA (`aten/src/ATen/cuda/`, `torch/cuda/`), MPS (`torch/mps/`), ROCm, and XPU backends from test scope -- Linux x86-64 (Ubuntu 22.04+ or equivalent CI runner) -- Python 3.11+ with a standard CPU-only PyTorch wheel (`pip install torch --index-url https://download.pytorch.org/whl/cpu`) -- No system-level dependencies beyond those bundled with the PyTorch wheel +- State the hardware scope the harness targets, and exclude everything out of scope (for example, CPU only, with GPU and other accelerator backends excluded). +- State the operating system and architecture the harness runs on. +- State the interpreter, runtime, or toolchain and how it is installed, preferring prebuilt artifacts over source builds for reproducibility. +- State any system-level dependencies beyond what the standard install provides. --- ## Benchmark Policy -- Each benchmark scenario must run for a minimum of **30 seconds wall-clock time** to yield stable CPU measurements (eliminates JIT warm-up and OS scheduling noise) -- Discard warm-up effects by one of two accepted methods: **(a)** an explicit warm-up phase of at least 5 iterations or 3 seconds (whichever is longer) before recording, or **(b)** reporting a warm-up-robust statistic — the **median** per-iteration latency (`p50`) over the full ≥ 30 s window, which absorbs the cold first iteration without a separate phase. The end-to-end baseline benchmark uses **(b)** (`median_iter_ms`); per-scenario tests may use either -- Report: mean, p50, p95, p99, and standard deviation over the measurement window -- Benchmarks must be isolated: no other benchmark may run concurrently in the same process -- Micro-benchmarks for dispatch-sensitive paths (< 1 μs per call) must use `timeit` or equivalent with ≥ 10,000 repetitions per measurement window +- Each benchmark scenario must run for a minimum wall-clock window long enough to yield stable measurements and absorb warm-up and scheduling noise. State the minimum. +- Discard warm-up effects by one of two accepted methods: **(a)** an explicit warm-up phase before recording, or **(b)** reporting a warm-up-robust statistic such as the median per-iteration latency over the full window, which absorbs the cold first iteration without a separate phase. +- Report a consistent set of statistics over the measurement window (for example mean, p50, p95, p99, and standard deviation). +- Benchmarks must be isolated: no other benchmark may run concurrently in the same process. +- Micro-benchmarks for latency-sensitive paths must use a repetition-based timer with enough repetitions per window to be stable. --- ## Machine Isolation (measurement environment) -End-to-end timing is only trustworthy on a quiesced, frequency-stable host. The baseline gate records a high coefficient of variation but does **not** fail on it, so these rules are what keep `cv_iter` low enough for the end-to-end measurement path to be usable — otherwise verdicts fall back to the per-op microbench path. +End-to-end timing is only trustworthy on a quiesced, frequency-stable host. If the baseline +gate records but does not fail on measurement variance, these rules are what keep variance +low enough for the end-to-end path to be usable; otherwise verdicts fall back to a +micro-benchmark path. -**The harness must implement (in `run.sh` / the benchmark process):** +**The harness must implement (in the benchmark process):** -- Pin the benchmark to a fixed set of dedicated physical cores when a core list is provided: honour a `BENCH_CPUSET` environment variable (e.g. `BENCH_CPUSET=2,3`) by launching the measurement under `taskset -c "$BENCH_CPUSET"` (or `numactl --physcpubind`). When it is unset, run unpinned but print a clear warning that results may be noisy. -- Set the BLAS/OpenMP thread count to **match the pinned set, not the whole machine**: `OMP_NUM_THREADS` / `MKL_NUM_THREADS` and `torch.set_num_threads()` must equal the number of pinned cores. Default to a fixed value, not `$(nproc)`, so the thread count is reproducible across hosts. -- Lower scheduling contention: run the measurement at a favourable priority (`nice`/`ionice` to the extent permitted without root) and start no concurrent work during a measurement window. -- Record the observed CPU state per run alongside the metrics — at minimum the mean CPU frequency and the 1-minute load average during the window — so a contended or throttled run is **visible in the report rather than silent**. +- Pin the benchmark to a fixed set of dedicated physical cores when a core list is provided (for example, honour a `BENCH_CPUSET` environment variable and launch the measurement pinned to it). When it is unset, run unpinned but print a clear warning that results may be noisy. +- Set any threading or parallelism controls to **match the pinned set, not the whole machine**, and default them to a fixed value rather than the host core count so the thread count is reproducible across hosts. +- Lower scheduling contention: run the measurement at a favourable priority to the extent permitted without elevated privileges, and start no concurrent work during a measurement window. +- Record the observed machine state per run alongside the metrics (at minimum the mean CPU frequency and the load average during the window) so a contended or throttled run is **visible in the report rather than silent**. **Environment preconditions (operator-provided; the harness cannot enforce these):** -- A dedicated, otherwise-idle host: no concurrent OS maintenance (on Windows hosts: Windows Update, Defender scans, search indexing, backup/sync), no other heavy processes, and no interactive use during collection. -- AC power, not battery — laptops drop to a power-saving frequency profile on battery. -- CPU governor set to `performance` and, for run-to-run consistency, turbo/boost disabled where permitted (`scaling_governor`, `intel_pstate/no_turbo`). Document it when the runner does not permit changing these. -- **Prefer native Linux or a dedicated cloud VM over WSL2 for any run whose end-to-end numbers are reported.** Under WSL2 the Windows host owns CPU frequency and thermals, so governor/turbo and core pinning cannot be fully enforced from inside the VM, and a laptop under sustained load will thermally throttle. +- A dedicated, otherwise-idle host: no concurrent OS maintenance (updates, security scans, indexing, backup or sync), no other heavy processes, and no interactive use during collection. +- AC power, not battery, since laptops drop to a power-saving frequency profile on battery. +- CPU governor set to a performance profile and, for run-to-run consistency, turbo or boost disabled where permitted. Document it when the runner does not permit changing these. +- Prefer native execution or a dedicated cloud VM over a nested/virtualized environment for any run whose end-to-end numbers are reported, because the outer host may own CPU frequency and thermals and prevent the governor and core pinning from being fully enforced. --- ## Suite Time Budget -- Total test suite (non-benchmark) runtime: < 10 minutes on a 4-core / 8 GB RAM CI runner -- Per-test timeout: 60 seconds (tests exceeding this are treated as hangs and fail the suite) -- Benchmark tests are excluded from the 60-second per-test cap but must individually complete within 5 minutes +- State the total non-benchmark suite runtime budget on a defined reference runner. +- State a per-test timeout above which a test is treated as a hang and fails the suite. +- State whether benchmark tests are exempt from the per-test cap and any separate per-benchmark limit. --- ## Isolation Requirements -- No external network calls during test execution (no model downloads, registry lookups, or remote fixture fetches) -- Each test must restore any global state it modifies before returning: - - `torch.set_default_dtype()` → restore original dtype - - `torch.manual_seed()` → document that seeding is intentional if used - - Registered hooks (forward hooks, backward hooks) → remove all hooks registered by the test - - `torch.backends.*` flags → restore original values -- No shared mutable tensor or `nn.Module` fixtures between tests (construct in-process per test) -- No temporary files left on the filesystem after a test completes +- No external network calls during test execution (no downloads, registry lookups, or remote fixture fetches). +- Each test must restore any global state it modifies before returning (for example default numeric precision, random seeds, registered hooks or callbacks, and any global backend flags). Document intentional seeding if used. +- No shared mutable fixtures between tests; construct per-test state in-process. +- No temporary files left on the filesystem after a test completes. --- ## Test Invocation -- All tests must be discoverable and runnable via: `pytest integration-test/tests/` -- No manual setup steps beyond: `pip install torch --index-url https://download.pytorch.org/whl/cpu` in a clean Python virtual environment -- No `conftest.py` may perform network I/O or download model weights -- Benchmark tests must be skippable via a standard marker: `pytest -m "not benchmark"` must run the non-benchmark suite +- All tests must be discoverable and runnable via a single documented command. +- No manual setup steps beyond the documented install into a clean environment. +- No test-configuration hook may perform network I/O or download large assets. +- Benchmark tests must be skippable via a standard marker so the non-benchmark suite can run on its own. diff --git a/constraints/project.md b/constraints/project.md index ebda1ca..e1f99a0 100644 --- a/constraints/project.md +++ b/constraints/project.md @@ -1,76 +1,64 @@ # Constraints -**Constraint:** Use PyTorch with CPU only. +This file pins the hard requirements for optimizing **your** target. CodeWeave does not +assume a language, runtime, or package manager, so everything specific to your codebase +lives here and the pipeline reads it from the generated manifest rather than hardcoding +it. Replace every placeholder below with your target's real values, and delete any section +that does not apply. -**Constraint (Python interpreter):** Build and run the harness against **Python 3.12**. -The venv MUST be created with the `python3.12` interpreter explicitly — never the ambient -`python3`. On this runner `python3` is a newer release (3.14) that has **no prebuilt PyPI -wheels** for several pinned harness dependencies (e.g. `tokenizers`), which forces source -builds that then fail against the runner's C23/GCC-15 toolchain (pyo3 ≤3.13, oniguruma -C). Python 3.12 is within PyTorch's supported range (`setup.py` `python_requires`) **and** -has full wheel coverage for the entire harness stack, so the install uses only prebuilt -wheels — no Rust/C compilation, and reproducible run-to-run. If `python3.12` is not on -PATH, the build script must **fail with the exact install command** (`apt-get install -y -python3.12 python3.12-venv python3.12-dev`) rather than silently using another interpreter. -> Adjust the version only to another PyTorch-supported release that also has full wheel -> coverage for the harness dependencies — confirm the interpreter is installed on the -> runner before changing. +Write each constraint as a single imperative statement prefixed with `**Constraint:**` so +the harness-design and build phases can parse them individually. Keep the "why" attached to +anything non-obvious; several of these categories exist because a repair loop rediscovered +the same fix more than once. -**Constraint:** When building PyTorch from source (Phase 5), the Python development -headers for the build interpreter must be installed before building (not discoverable from -CONTRIBUTING.md): `python3.12-dev` (provides `Python.h`; without it CMake cannot build -`torch._C` even when `BUILD_PYTHON=1` is set — it silently falls back to `BUILD_PYTHON=OFF`). -Keep the `-dev` package version matched to the interpreter above. +**Constraint (runtime and scope):** State the runtime, backend, and hardware scope the +analysis targets, and exclude everything out of scope. Example shape: "Target the CPU +inference path only; exclude GPU, distributed, and quantization backends." Keeping the +scope narrow makes the baseline reproducible and the hotspot search tractable. -**Constraint:** When building PyTorch from source (Phase 5), the following environment -variables must be set before running `pip install -e .`. These are not all discoverable -from CONTRIBUTING.md — apply them unconditionally: +**Constraint (interpreter or toolchain version):** Pin the exact interpreter, compiler, or +SDK version the harness builds and runs against, and say how to obtain it. Pin a version +that has full prebuilt-artifact coverage for your dependency stack so the install does not +fall back to source builds against an unexpected toolchain. If the required version is not +present, the build script should fail with the exact install command rather than silently +using whatever is on PATH. + +**Constraint (build prerequisites):** List any development headers, system libraries, or +build tools that must be installed before the target builds, especially ones not documented +in the target's own contributing guide. Missing prerequisites often fail silently by +disabling a component rather than erroring, so name each one and the symptom of its absence. + +**Constraint (build configuration):** If the target is built from source, list the exact +build flags or environment variables to set and why each one matters. Prefer a table so the +build phase applies them unconditionally: | Variable | Value | Why | |---|---|---| -| `BUILD_PYTHON` | `1` | CMake defaults this to OFF; without it `libtorch_python.so` and `torch._C` are not compiled, making the package unimportable | -| `BUILD_TEST` | `0` | Skip building test binaries (~30% of build time) | -| `USE_CUDA` | `0` | CPU-only build | -| `USE_DISTRIBUTED` | `0` | Not needed for inference harness | -| `USE_FBGEMM` | `0` | Quantisation backend not required | -| `USE_NNPACK` | `0` | Not required | -| `USE_QNNPACK` | `0` | Not required | -| `USE_XNNPACK` | `0` | Not required | -| `USE_FLASH_ATTENTION` | `0` | CUDA-only feature | -| `USE_MEM_EFF_ATTENTION` | `0` | CUDA-only feature | +| `EXAMPLE_FLAG` | `1` | What breaks or slows down if it is left at its default | + +Favor flags that disable out-of-scope features (GPU, distributed, optional backends) to cut +build time and shrink the surface the harness has to stub. **Constraint:** Do not perform any GIT commits. These will be handled externally. -**Constraint:** Use codecarbon for energy measurement in integration tests (`pip install codecarbon`; wrap benchmark runs with `EmissionsTracker`). +**Constraint (energy measurement):** Name the energy or emissions measurement tool the +integration tests must use and how to wire it in, so every benchmark run is instrumented the +same way. This is the project's core signal; do not leave it implicit. -**Constraint (harness venv must be able to collect PyTorch's own test suite):** -`integration-test/requirements.txt` must include `expecttest` and `hypothesis`. The -Phase 7 correctness gates run PyTorch's own tests (`src/test/...`, and -`test/test_ops.py` for the OpInfo gate) with the harness venv interpreter, and -`torch.testing._internal.common_utils` unconditionally does `import expecttest` -(several suites also use `hypothesis`). These packages are listed in PyTorch's -`src/requirements.txt` under "Install / Development extra requirements" — they are -NOT in `requirements-build.txt`, so a build-requirements-only install leaves the venv -unable to even collect the test suite, and every optimization cycle fails its gates -with `ModuleNotFoundError: expecttest` regardless of the change under test (observed: -14 of 15 iteration failures in one run). +**Constraint (test dependencies):** If the correctness gates run the target's own test +suite, list every package that suite needs to even be collected, not just to pass. Test +harnesses frequently import helper libraries at module load time, so a +build-requirements-only install can leave the venv unable to collect the suite at all, which +fails every optimization cycle regardless of the change under test. -**Constraint (conftest must stub the uncompiled distributed modules):** because the -build uses `USE_DISTRIBUTED=0`, `transformers.integrations.fsdp.is_fsdp_managed_module()` -(called during `model.generate()`) triggers `import torch.distributed.fsdp` → -`torch.testing._internal.distributed.fake_pg` → `torch._C._distributed_c10d`, which is -not compiled — an `ImportError` at inference time. The harness `tests/conftest.py` MUST -insert `sys.modules` stubs **before anything imports torch**: (1) a stub module for -`torch._C._distributed_c10d` exposing no-op `FakeProcessGroup` and `FakeStore` classes; -(2) a stub module for `torch.distributed.fsdp` exposing a no-op -`FullyShardedDataParallel` class; then, after `import torch.distributed`, bind the fsdp -stub as an attribute of `torch.distributed`. Do NOT replace the whole -`torch.distributed` module — partial replacement breaks other attribute access (e.g. -`torch.distributed.Backend`). This fix has been rediscovered by repair loops in two -separate runs; generate it up front. +**Constraint (harness shims for disabled features):** If your build configuration disables a +feature that the target still imports at runtime, describe the shim the harness must install +before anything imports the target, and describe it precisely enough to generate up front. +Partial stubs are usually safer than replacing a whole module, because downstream code often +reaches for unrelated attributes on the same module. This kind of fix tends to be +rediscovered by repair loops, so writing it here once saves cycles. -**Constraint (transformers version):** pin `transformers` to a release verified to -import and run `generate()` against this CPU-only, `USE_DISTRIBUTED=0` source build -with the conftest stubs above. `transformers==4.47.1` is verified (archived -`codeweave-run` branch); prefer it over nearby releases unless the chosen version has -been re-verified against the build. \ No newline at end of file +**Constraint (dependency version pins):** Pin any surrounding dependency (framework, model +runner, data library) to a release verified to import and run against your build +configuration, and record where that verification happened. Prefer a version you have +confirmed over a newer one you have not.