Skip to content

fix: validate the project before generate.py deletes notebooks/ - #216

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/generate-validate-before-rmtree
Jul 30, 2026
Merged

fix: validate the project before generate.py deletes notebooks/#216
Jammy2211 merged 1 commit into
mainfrom
feature/generate-validate-before-rmtree

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Closes #215.

What was broken

generate.py cleared the entire notebooks/ tree at line 114, but the only rejection of an unknown project was inject_colab_setup's raise — reached inside the per-script loop six lines later. Running it on an unregistered project deleted 113 tracked notebooks in autocti_workspace and then aborted, leaving a stray intermediate .ipynb inside scripts/. Only git checkout recovered it; in a repo with uncommitted notebook work, nothing would.

The root-level start_here*.py loop (generate.py:83-91) does inject before the rmtree, so a workspace with a root start_here.py failed safely. autocti_workspace has none. The safety was accidental, not designed — which is why this survived: the repos anyone normally runs this on all have a root start_here.py.

Three separate sessions hit this independently before it got fixed.

The fix

  1. Validate up front, before generate_project_folders() and the rmtree, against build_util.COLAB_PROJECTS — kept as the single source of truth rather than duplicating the project list. sys.exit matches the existing guard style in generate_markdown.py:450. inject_colab_setup's own raise stays as the backstop for direct callers.
  2. A finally on the per-script loop that removes the intermediate notebook, so a mid-loop failure cannot strand an untracked .ipynb inside scripts/.

Verification

Tests are subprocess-driven: generate.py parses argv and does all its work at module level under __main__, so import cannot exercise it. The fixture is a real git init workspace with a tracked notebooks/ tree.

  • pytest tests/ -q258 passed.
  • Both ordering tests fail with the guard reverted (git stash push -- autohands/generate.py), so they pin the bug rather than passing vacuously. The third — a known project is still accepted — passes either way by design, guarding against the check rejecting valid input.

Live, on the real repos:

check before after
generate.py autocti from autocti_workspace/ 113 staged deletions + a stray scripts/dataset_1d/advanced/database/examples/data_fitting.ipynb exit 1, "Nothing was modified", git status empty, 79 notebooks intact, 0 stray
generate.py howtolens from HowToLens/ exit 0, zero diff

Deliberately out of scope

Registering autocti for Colab, the PyAutoNerves _PROJECTS entry, pre_build.sh wiring, and regenerating autocti's 79 notebooks are owned by PyAutoMind/draft/release/autocti/cti_release_train_wiring.md (human-required), whose item 3 is exactly "generate.py autocti (blocked today by the COLAB_PROJECTS registry check) → commit regenerated notebooks", and whose item 2 carries the arcticpy recipe (apt libgsl-dev, pip install numpy cython, arcticpy==2.6 --no-build-isolation --no-deps — a naive install downgrades numpy below 2.0).

So autocti_workspace/notebooks/ keeps its 34 Finish. cells and 4 mangled code cells until that task runs. It regenerates all 79 notebooks anyway, so clearing them here would be throwaway work — and declaring autocti "deliberately non-Colab" to unblock generation would contradict that task's intent. What changes today is that the blocker is now a clean refusal instead of a destructive one.

🤖 Generated with Claude Code

https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM

`generate.py` cleared the whole `notebooks/` tree at line 114, but the only
rejection of an unknown project was `inject_colab_setup`'s raise, reached inside
the per-script loop six lines later. Running it on an unregistered project
therefore deleted 113 tracked notebooks in autocti_workspace and *then* aborted,
leaving a stray intermediate .ipynb inside scripts/. Only `git checkout`
recovered it — and in a repo with uncommitted notebook work, nothing would.

The root-level start_here*.py loop does inject before the rmtree, so a workspace
with a root start_here.py failed safely; autocti_workspace has none. The safety
was accidental, not designed.

Validate up front against `build_util.COLAB_PROJECTS` (kept as the single source
of truth) and exit before touching anything. `inject_colab_setup`'s raise stays
as the backstop for direct callers. Also give the per-script loop a `finally`
that removes the intermediate notebook, so a mid-loop failure cannot strand an
untracked .ipynb in scripts/.

Tests are subprocess-driven — generate.py parses argv and works at module level
under __main__, so import cannot exercise it. Both ordering tests fail with the
guard reverted; the third pins that a valid project is still accepted.

Closes #215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM
@Jammy2211
Jammy2211 merged commit 28106fa into main Jul 30, 2026
@Jammy2211
Jammy2211 deleted the feature/generate-validate-before-rmtree branch July 30, 2026 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: generate.py deletes notebooks/ before validating the project name

1 participant