feat(hygiene): hygiene conductor — phase 3 perf mode (#91)

agents/conductors/hygiene/AGENTS.md

@@ -19,28 +19,29 @@ paired repo**, boundaries) recorded in PyAutoMind
 
 ## Modes
 
-Each mode does a cheap, read-only local **pre-scan** and **delegates** the full
-audit + any execution to the owning skill — the conductor never runs a heavy
-audit and never mutates a repo. A pre-scan is one of three kinds, which is what
-makes its count comparable (or not):
+All five modes are live. Each does a cheap, read-only local **pre-scan** and
+**delegates** the full audit + any execution to the owning skill — the conductor
+never runs a heavy audit and never mutates a repo. A pre-scan is one of a few
+kinds, which is what makes its count comparable (or not):
 
 - **debris** — finds directly-removable items; a real, rankable count (`tidy`).
+- **timing** — measures import cost; a real, rankable count of *slow* imports (`perf`).
 - **surface** — only *sizes* the audit; the real problems emerge when the
   delegated skill runs, so the count is **not** a problem count (`deps`, `docs`).
 - **advisory** — no cheap local signal at all (`noise`).
 
 | Mode | Pre-scan (kind) | Delegates to |
 |------|-----------------|--------------|
-| `perf` | *staged — phase 3* (dev-loop timing: slow tests / integration-mode scripts / imports) | `refactor` / `bug` |
+| `perf` | import cost — times `import <pkg>` per library in a **subprocess** (**timing**); heavy test/script timing is read from Heart's `script_timing` / `test_run` | `/refactor` / `/bug` (+ Heart timing legs) |
 | `tidy` | git debris — stale branches, stashes, `[gone]` refs, dirty checkouts (**debris**) | `/repo_cleanup` (Brain) |
 | `noise` | none — needs a pytest + workspace-script run (**advisory**) | `/cli_noise_clean` (Heart) |
 | `deps` | capped/pinned specifiers in library `pyproject.toml` (**surface**) | `/dep_audit` (Heart, hits PyPI) |
 | `docs` | `docs/api/*.rst` + `currentmodule` counts across the 3 doc repos (**surface**) | `/audit_docs` (Heart, imports) |
-| *(default)* | all of the above | a ranked `HygieneDecision` worklist — recommends `tidy` when debris exists (the only directly-actionable count), else prompts the periodic audits |
+| *(default)* | all of the above (**perf timing deferred** — it spawns real imports) | a ranked `HygieneDecision` worklist — recommends `tidy` when debris exists, then `hygiene perf`, then the periodic audits |
 
 ```
-pyauto-brain hygiene              # audit across modes → ranked worklist
-pyauto-brain hygiene perf         # dev-loop timing        (staged: phase 3)
+pyauto-brain hygiene              # pre-scan across modes → ranked worklist
+pyauto-brain hygiene perf         # import cost (subprocess) → /refactor + Heart legs
 pyauto-brain hygiene tidy         # git debris → /repo_cleanup
 pyauto-brain hygiene noise        # CLI noise → /cli_noise_clean
 pyauto-brain hygiene deps         # dependency-cap surface → /dep_audit
@@ -50,8 +51,12 @@ pyauto-brain hygiene <mode> --json
 
 Repos are read under `PYAUTO_ROOT` (default `~/Code/PyAutoLabs`). `noise`/`deps`/
 `docs` route to **read-only PyAutoHeart observation skills** — measurement lives
-in Heart; hygiene pre-scans, prioritises and routes. `perf` and any new standing
-Heart legs (`import_time` / `cli_noise`) remain phase 3.
+in Heart; hygiene pre-scans, prioritises and routes. `perf` times imports in a
+**subprocess** (`HYGIENE_PYTHON`, default `python3` — point it at the PyAuto venv
+to time the science libs), so the conductor itself never imports the JAX stack;
+the slow-test / slow-script signal is read from Heart, not re-run. A *standing*
+Heart `import_time` (or `cli_noise`) leg — promoting the import pre-scan to a
+tracked Heart signal — is a deferred optional follow-up (a PyAutoHeart change).
 
 ## Fundamental principles
 

agents/conductors/hygiene/hygiene.sh

@@ -10,23 +10,28 @@
 # never mutates a repo, and (like profiling) it stays stdlib/bash so it never
 # drags the JAX stack into the Brain.
 #
-# Each mode does a cheap, read-only local PRE-SCAN (signals bash can compute in
-# seconds) and DELEGATES the full audit + any execution to the owning skill:
-# tidy -> /repo_cleanup (Brain), noise -> /cli_noise_clean, deps -> /dep_audit,
-# docs -> /audit_docs (the last three are read-only PyAutoHeart observation
-# skills — measurement lives in Heart; hygiene routes and prioritises).
+# Each mode does a cheap, read-only local PRE-SCAN and DELEGATES the full audit +
+# any execution to the owning skill:
+#   tidy  -> /repo_cleanup (Brain)         perf  -> /refactor (+ Heart timing legs)
+#   noise -> /cli_noise_clean (Heart)      deps  -> /dep_audit (Heart)
+#   docs  -> /audit_docs (Heart)
+# The three Heart skills are read-only observation skills — measurement lives in
+# Heart; hygiene routes and prioritises. perf's import timing runs in a
+# SUBPROCESS, so the conductor itself never imports the science/JAX stack.
 #
 # Usage:
-#   hygiene.sh                 # audit across modes -> prioritised worklist (default)
-#   hygiene.sh perf            # dev-loop timing: unit tests / integration scripts / imports (phase 3)
+#   hygiene.sh                 # pre-scan across modes -> ranked worklist (default)
+#   hygiene.sh perf            # import-cost timing (subprocess) -> /refactor + Heart legs
 #   hygiene.sh tidy            # git debris pre-scan -> /repo_cleanup
 #   hygiene.sh noise           # CLI-noise route -> /cli_noise_clean
 #   hygiene.sh deps            # dependency-cap pre-scan -> /dep_audit
 #   hygiene.sh docs            # API-docs pre-scan -> /audit_docs
 #   hygiene.sh <mode> --json   # machine-readable HygieneDecision
 #
-# perf stays staged for phase 3; the other four modes are live (pre-scan +
-# delegate). The conductor reads repos under PYAUTO_ROOT (default ~/Code/PyAutoLabs).
+# All five modes are live. The fast default scan DEFERS perf's import timing (it
+# spawns real imports); run `hygiene perf` for it. Repos are read under
+# PYAUTO_ROOT (default ~/Code/PyAutoLabs); import timing uses HYGIENE_PYTHON
+# (default python3 — point it at the PyAuto venv to time the science libs).
 
 set -uo pipefail
 
@@ -40,21 +45,30 @@ LIB_REPOS=(PyAutoConf PyAutoFit PyAutoArray PyAutoGalaxy PyAutoLens)
 ORG_REPOS=(PyAutoBrain PyAutoBuild PyAutoHeart PyAutoMind)
 DOC_REPOS=(PyAutoFit PyAutoGalaxy PyAutoLens)
 
+# perf: import timing is measured in a subprocess with this interpreter (never
+# imported into the conductor). Point HYGIENE_PYTHON at the PyAuto venv to time
+# the science libs; HYGIENE_PERF_LIBS overrides the import names (tests use
+# fast stdlib modules); HYGIENE_PERF_THRESHOLD (s) is the slow cutoff.
+PERF_PY="${HYGIENE_PYTHON:-python3}"
+PERF_THRESHOLD="${HYGIENE_PERF_THRESHOLD:-3.0}"
+read -r -a PERF_LIBS <<< "${HYGIENE_PERF_LIBS:-autoconf autofit autoarray autogalaxy autolens}"
+
 MODE_ORDER=(perf tidy noise deps docs)
 declare -A MODE_DELEGATE=(
-  [perf]="(phase 3 — not yet implemented)"
+  [perf]="/refactor"
   [tidy]="/repo_cleanup"
   [noise]="/cli_noise_clean"
   [deps]="/dep_audit"
   [docs]="/audit_docs"
 )
-# A mode's pre-scan is one of three kinds, which is what makes its count
-# comparable (or not): 'debris' pre-scans find directly-removable items (a real,
-# rankable count); 'surface' pre-scans only size the audit (the actual problems
-# emerge only when the delegated skill runs — the count is NOT a problem count);
-# 'advisory' has no cheap local signal. Only 'debris' counts drive the ranking.
+# A mode's pre-scan is one of a few kinds, which is what makes its count
+# comparable (or not): 'debris' finds directly-removable items and 'timing'
+# finds slow imports — both real, rankable counts; 'surface' only sizes the
+# audit (the real problems emerge when the delegated skill runs — the count is
+# NOT a problem count); 'advisory' has no cheap local signal. Only 'debris' and
+# 'timing' counts drive the ranking.
 declare -A MODE_KIND=(
-  [tidy]="debris" [deps]="surface" [docs]="surface" [noise]="advisory"
+  [perf]="timing" [tidy]="debris" [deps]="surface" [docs]="surface" [noise]="advisory"
 )
 
 # --- Pre-scan helpers (read-only; each echoes "count|one-line summary"). -------
@@ -114,9 +128,34 @@ prescan_noise() {
   echo "-1|no cheap local signal — runs pytest + workspace scripts (PYAUTO_TEST_MODE=2)"
 }
 
+# perf: import-cost timing — time `import <pkg>` per library in a SUBPROCESS
+# (best-effort; the conductor never imports the science stack itself). The count
+# is the number of libraries whose import exceeds the slow threshold. Heavy
+# dev-loop timing (slow tests / integration scripts) is already observed by
+# PyAutoHeart's script_timing / test_run legs — perf points there and routes.
+prescan_perf() {
+  local slow=0 measured=0 detail="" pkg rc start end t
+  for pkg in "${PERF_LIBS[@]}"; do
+    [[ -n "$pkg" ]] || continue
+    start=$(date +%s.%N)
+    timeout 60 "$PERF_PY" -c "import ${pkg}" >/dev/null 2>&1; rc=$?
+    end=$(date +%s.%N)
+    if [[ $rc -ne 0 ]]; then detail+="${pkg}:n/a "; continue; fi
+    measured=$((measured + 1))
+    t=$(awk "BEGIN{printf \"%.2f\", ${end}-${start}}")
+    detail+="${pkg}:${t}s "
+    awk "BEGIN{exit !(${t} > ${PERF_THRESHOLD})}" && slow=$((slow + 1))
+  done
+  if [[ $measured -eq 0 ]]; then
+    echo "-1|no library importable here (set HYGIENE_PYTHON to the PyAuto venv) — slow tests/scripts live in Heart (script_timing/test_run)"
+  else
+    echo "${slow}|${measured} libs timed, >${PERF_THRESHOLD}s = slow: ${detail}(slow tests/scripts: see Heart script_timing/test_run)"
+  fi
+}
+
 prescan() {
   case "$1" in
-    tidy) prescan_tidy ;; deps) prescan_deps ;;
+    perf) prescan_perf ;; tidy) prescan_tidy ;; deps) prescan_deps ;;
     docs) prescan_docs ;; noise) prescan_noise ;;
   esac
 }
@@ -139,18 +178,23 @@ if [[ "$mode" == "help" ]]; then
   exit 0
 fi
 
+# perf's import timing spawns real imports, so the fast default scan defers it;
+# an explicit `hygiene perf` runs it. This predicate decides which.
+perf_deferred() { [[ "$1" == "perf" && "$mode" == "default" ]]; }
+
 # --- JSON footing: a HygieneDecision the Brain session can consume. ------------
 emit_json_row() { # mode
   local m="$1"
-  if [[ "$m" == "perf" ]]; then
-    printf '{"mode":"perf","status":"staged","lands":"phase 3"}'; return
+  if perf_deferred "$m"; then
+    printf '{"mode":"perf","status":"deferred","hint":"run: pyauto-brain hygiene perf (import timings; skipped in the fast default scan)","delegate":"/refactor"}'
+    return
   fi
   local res count summary kind status
   res="$(prescan "$m")"; count="${res%%|*}"; summary="${res#*|}"; kind="${MODE_KIND[$m]}"
-  if   [[ "$kind" == "advisory" ]]; then status="advisory"
-  elif [[ "$kind" == "surface"  ]]; then status="surface"
-  elif [[ "$count" == "0"       ]]; then status="clean"
-  else status="debris"; fi
+  if   [[ "$kind" == "advisory" || "$count" == "-1" ]]; then status="advisory"
+  elif [[ "$kind" == "surface" ]]; then status="surface"
+  elif [[ "$count" == "0" ]]; then status="clean"
+  else status="$kind"; fi   # debris | timing
   printf '{"mode":"%s","kind":"%s","status":"%s","count":%s,"summary":"%s","delegate":"%s"}' \
     "$m" "$kind" "$status" "$([[ "$count" == "-1" ]] && echo null || echo "$count")" \
     "${summary//\"/\\\"}" "${MODE_DELEGATE[$m]}"
@@ -174,35 +218,45 @@ echo "The hygiene conductor pre-scans code-quality debt (read-only) and delegate
 echo "audit + fix to the owning skill — it never mutates a repo itself."
 echo
 
+render_delegate_line() { # mode
+  local m="$1"
+  if [[ "${MODE_KIND[$m]}" == "timing" ]]; then
+    printf '  %-6s %-9s → route slow items to %s; slow tests/scripts → Heart script_timing/test_run\n' "" "" "${MODE_DELEGATE[$m]}"
+  else
+    printf '  %-6s %-9s → run %s for the full audit\n' "" "" "${MODE_DELEGATE[$m]}"
+  fi
+}
+
 render_row() { # mode
   local m="$1"
-  if [[ "$m" == "perf" ]]; then
-    printf '  %-6s %-9s %s\n' "perf" "staged" "dev-loop timing (phase 3) — not yet implemented"
+  if perf_deferred "$m"; then
+    printf '  %-6s %-9s %s\n' "perf" "run it" "import timings (subprocess) — run 'hygiene perf'; deferred in the fast default scan"
+    render_delegate_line "$m"
     return
   fi
   local res count summary kind tag
   res="$(prescan "$m")"; count="${res%%|*}"; summary="${res#*|}"; kind="${MODE_KIND[$m]}"
-  if   [[ "$kind" == "advisory" ]]; then tag="advisory"
-  elif [[ "$kind" == "surface"  ]]; then tag="surface"
-  elif [[ "$count" == "0"       ]]; then tag="clean"
+  if   [[ "$kind" == "advisory" || "$count" == "-1" ]]; then tag="advisory"
+  elif [[ "$kind" == "surface" ]]; then tag="surface"
+  elif [[ "$count" == "0" ]]; then tag="clean"
+  elif [[ "$kind" == "timing" ]]; then tag="${count} slow"
   else tag="${count} debris"; fi
   printf '  %-6s %-9s %s\n' "$m" "$tag" "$summary"
-  printf '  %-6s %-9s → run %s for the full audit\n' "" "" "${MODE_DELEGATE[$m]}"
+  render_delegate_line "$m"
 }
 
 if [[ "$mode" == "default" ]]; then
-  # Only 'debris' pre-scans yield a directly-actionable count, so the ranking is
-  # honest: recommend tidy when there is removable debris; otherwise the
-  # 'surface'/'advisory' modes are periodic audits (their real findings only
-  # emerge when the delegated skill runs — the pre-scan can't rank them).
+  # Only the 'debris'/'timing' pre-scans yield a directly-actionable count, and
+  # perf's timing is deferred here (too slow for the fast scan) — so the default
+  # recommendation ranks on tidy debris and points at the periodic audits.
   tidy_n="$(prescan tidy)"; tidy_n="${tidy_n%%|*}"
   for m in "${MODE_ORDER[@]}"; do render_row "$m"; done
   echo
   if [[ "$tidy_n" -gt 0 ]]; then
     echo "Recommended next: hygiene tidy (${tidy_n} removable debris items), then run /repo_cleanup."
-    echo "  deps/docs report audit *surface*, not problems — run their skills periodically to surface real findings."
+    echo "  Then 'hygiene perf' for import timings; deps/docs/noise are periodic audits (surface only)."
   else
-    echo "Recommended next: no removable debris — run deps/docs/noise audits periodically (their pre-scans size the surface only)."
+    echo "Recommended next: no removable debris — run 'hygiene perf' for import timings, and deps/docs/noise audits periodically."
   fi
   echo "Design: PyAutoMind research/pyautobrain/hygiene_agent_decision.md."
 else

skills/hygiene/hygiene.md

@@ -9,18 +9,20 @@ Shared routing context: `PyAutoBrain/skills/COMMANDS.md`.
 
 ## Do
 
-1. Run `bin/pyauto-brain hygiene [tidy | noise | deps | docs]` (no arg = pre-scan
-   across modes → a ranked worklist). This is a **dry run** — each mode does a
-   cheap read-only pre-scan and emits a `HygieneDecision` naming the skill to run
-   for the full audit. Nothing is executed or mutated.
+1. Run `bin/pyauto-brain hygiene [perf | tidy | noise | deps | docs]` (no arg =
+   pre-scan across modes → a ranked worklist; perf's import timing is deferred
+   there). This is a **dry run** — each mode does a cheap read-only pre-scan and
+   emits a `HygieneDecision` naming the skill to run for the full audit. Nothing
+   is executed or mutated.
 2. Execute the emitted plan: run the named delegate — `/repo_cleanup` (git
    debris), `/cli_noise_clean`, `/dep_audit`, `/audit_docs` — for the full audit,
-   then route any code fixes to `/refactor` / `/bug` / `/feature`, shipped via
-   `ship_library` / `ship_workspace`.
+   or for `perf` route slow imports/functions to `/refactor` / `/bug` (JAX-adapt
+   is a judgement call, never automatic), shipped via `ship_library` / `ship_workspace`.
 
 The Hygiene Agent **reasons; it never edits source and never mutates a repo.**
 Measurement lives in Heart (`noise`/`deps`/`docs` route to read-only PyAutoHeart
-skills, plus the `script_timing` / `test_run` signals); hygiene pre-scans + routes.
+skills; `perf`'s slow-test/script signal is read from Heart's `script_timing` /
+`test_run`, and its import timing runs in a subprocess); hygiene pre-scans + routes.
 
-> **Staged:** only `perf` (dev-loop timing) remains staged — it lands in phase 3
-> with any new PyAutoHeart legs. `tidy` / `noise` / `deps` / `docs` are live.
+All five modes are live. Point `HYGIENE_PYTHON` at the PyAuto venv for `perf` to
+time the science libraries (otherwise it reports advisory).

tests/test_hygiene_conductor.py

@@ -15,8 +15,10 @@
 MODES = {"perf", "tidy", "noise", "deps", "docs"}
 
 
-def _run(args, root):
+def _run(args, root, extra=None):
     env = {**os.environ, "PYAUTO_ROOT": str(root)}
+    if extra:
+        env.update(extra)
     return subprocess.run(
         [str(BRAIN), "hygiene", *args],
         capture_output=True, text=True, env=env,
@@ -30,11 +32,16 @@ def test_default_json_is_a_hygiene_decision_with_all_modes(tmp_path):
     assert doc["decision"] == "HygieneDecision"
     assert doc["mode"] == "default"
     assert {row["mode"] for row in doc["rows"]} == MODES
-    # perf is staged; the other four carry a pre-scan kind.
+    # the four pre-scan modes carry their kind; perf's timing is deferred in the
+    # fast default scan (it spawns real imports).
     kinds = {row["mode"]: row.get("kind") for row in doc["rows"]}
     assert kinds["tidy"] == "debris"
     assert kinds["deps"] == "surface" and kinds["docs"] == "surface"
     assert kinds["noise"] == "advisory"
+    perf = next(row for row in doc["rows"] if row["mode"] == "perf")
+    assert perf["status"] == "deferred"
+    # nothing is staged any more — all five modes are live.
+    assert all(row.get("status") != "staged" for row in doc["rows"])
 
 
 def test_single_mode_json_round_trips(tmp_path):
@@ -47,6 +54,25 @@ def test_single_mode_json_round_trips(tmp_path):
         assert doc["row"]["delegate"].startswith("/")
 
 
+def test_perf_times_imports_in_a_subprocess(tmp_path):
+    # Fast stdlib modules keep the test hermetic + quick; the point is the row
+    # shape, not the science libs (which need the PyAuto venv).
+    r = _run(["perf", "--json"], tmp_path, extra={"HYGIENE_PERF_LIBS": "sys json"})
+    assert r.returncode == 0, r.stderr
+    row = json.loads(r.stdout)["row"]
+    assert row["mode"] == "perf"
+    assert row["kind"] == "timing"
+    assert row["delegate"] == "/refactor"
+    assert row["status"] in {"clean", "timing"}  # sys/json import well under threshold
+
+
+def test_perf_advisory_when_nothing_importable(tmp_path):
+    r = _run(["perf", "--json"], tmp_path, extra={"HYGIENE_PERF_LIBS": "nope_not_a_module_xyz"})
+    assert r.returncode == 0, r.stderr
+    row = json.loads(r.stdout)["row"]
+    assert row["status"] == "advisory" and row["count"] is None
+
+
 def test_unknown_mode_exits_2(tmp_path):
     r = _run(["bogus"], tmp_path)
     assert r.returncode == 2