feat(clean_slate): write-site dataset provenance, tests, gc phase, honest wake_up claim

[Jammy2211]

Overview

bin/clean_slate.sh's #167 provenance sweep misses ~6 MB of regenerated dataset cruft because its rule (name+type literals in a simulator* file) cannot see non-simulator writers — the HowTo repos are a 100% miss (their entire live dataset payload is tutorial-written), and start_here.py-written dirs in both workspaces survive every run. Phases 1/1b/2/3 have zero test coverage, the size warning spams per file with no dedup, and wake_up.md claims the script is "non-destructive … reversible" while phase 1b rm -rfs untracked dirs git cannot restore. Leg 5 (final) of the PyAutoMind dataset-bulk series. Closes the series' machinery goal: the wake_up cleanup reclaims regenerated bulk across all covered repos, safely, with tests.

Design (decided)

• New bin/dataset_provenance.py (python3, stdlib-only): given a repo root and candidate dataset/<type>/<name> dirs, classify each by write site, never name-mention: REGENERABLE (some script constructs the dataset path and calls an output/write API on it — output_to_fits, output_to_json, dataset.output_to_fits, etc. — and no download markers write into it), DOWNLOADED (urllib/urlopen/urlretrieve/requests feed it → never delete), ORPHAN (no writer → report, don't delete). Phase 1b calls it, replacing the grep rule; the bash-side rails stay (untracked-only, depth-3, tracked-file guard) and empty dataset dirs are removed.
• Size warning: aggregate per dataset dir (one line per dir, not per file).
• New git gc --auto phase per repo (HowTo repos had 42 MB of never-packed loose objects before the 2026-07-27 one-off gc; this stops regrowth).
• .ipynb_checkpoints swept; __pycache__ deliberately left (import-speed cache, already gitignored) — recorded here so the decision is findable.
• wake_up.md:18-21 reworded: phase 1b deletes untracked regenerable data (recreated on demand), it is not "reversible".
• Net-new pytest coverage (fixture git repos): never deletes tracked files; deletes untracked simulator-written AND start_here-written dirs; NEVER deletes download-written dirs; sdp81-style file that reads real data by name while writing a simulated dir → real kept, written deleted; orphan reported not deleted; per-dir size warning; empty-dir removal; gc phase runs. tests/test_hygiene_conductor.py re-checked if the CLI surface changes.

Ground truth for the rule (verified 2026-07-27, in the Mind prompt)

Must newly catch: autolens interferometer/simulated_lens (5.0 MB), imaging/simulated_lens, point_source/{simulated_lens,start_here_example}, cluster/csv_api_example; autogalaxy imaging/simulated_galaxy (772 KB), interferometer/simulated_galaxy; HowToLens imaging/howtolens; HowToGalaxy imaging/howtogalaxy. Must never touch: cluster/smacs0723 (52 MB GPL download), cluster/a2744/data.fits, weak/a2744_pyrrg, multi/rxj1131 (the last two are name-mentioned in scripts — the write-site discrimination is what protects them). Must report as orphan: autolens imaging/tutorial (188 KB, no writer anywhere).

Branch survey

Repository	Current Branch	Dirty?
./PyAutoBrain	main	clean

Branch: feature/clean-slate-write-site — Worktree: ~/Code/PyAutoLabs-wt/clean-slate-write-site/ — Classification: Library-ish (Brain organ)

[Jammy2211]

Shipped to PR #170 (awaiting human merge — the final leg of the dataset-bulk series).

What landed

• bin/dataset_provenance.py — AST write-site classifier (REGENERABLE / DOWNLOADED / ORPHAN); deletion requires positive write evidence, reads cannot launder into writes, network-binding files blanket-protect their datasets, orphans are reported never deleted.
• clean_slate phase 1b delegates to it (hard-fail, no fallback), prunes emptied dirs, per-dir size warnings, .ipynb_checkpoints sweep (__pycache__ deliberately kept), per-repo git gc --auto.
• wake_up.md guardrail wording corrected (recoverable-by-regeneration, not 'reversible').

Verification

• clean_slate tests 2 → 14 (phases 1/1b/2/3 previously had zero coverage), incl. the sdp81 read-vs-write doctrine case and download protection; full suite 178 passed (1 pre-existing deselect: the concurrent sizing-faculty wrapper gap, unrelated).
• Real-workspace DRY_RUN acceptance: newly catches the start_here/tutorial-written cruft across both workspaces and the HowTo repos (~8 MB today); dataset/imaging/tutorial reported as orphan; none of the never-touch set (smacs0723, a2744, weak/a2744_pyrrg, multi/rxj1131, sdp81, uv_wavelengths) classifies as regenerable.

Shipped under the 2026-07-27 Heart YELLOW acknowledgement recorded in PyAutoMind active.md.