fix: validate the project before generate.py deletes notebooks/ - #216
Merged
Conversation
`generate.py` cleared the whole `notebooks/` tree at line 114, but the only rejection of an unknown project was `inject_colab_setup`'s raise, reached inside the per-script loop six lines later. Running it on an unregistered project therefore deleted 113 tracked notebooks in autocti_workspace and *then* aborted, leaving a stray intermediate .ipynb inside scripts/. Only `git checkout` recovered it — and in a repo with uncommitted notebook work, nothing would. The root-level start_here*.py loop does inject before the rmtree, so a workspace with a root start_here.py failed safely; autocti_workspace has none. The safety was accidental, not designed. Validate up front against `build_util.COLAB_PROJECTS` (kept as the single source of truth) and exit before touching anything. `inject_colab_setup`'s raise stays as the backstop for direct callers. Also give the per-script loop a `finally` that removes the intermediate notebook, so a mid-loop failure cannot strand an untracked .ipynb in scripts/. Tests are subprocess-driven — generate.py parses argv and works at module level under __main__, so import cannot exercise it. Both ordering tests fail with the guard reverted; the third pins that a valid project is still accepted. Closes #215 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #215.
What was broken
generate.pycleared the entirenotebooks/tree at line 114, but the only rejection of an unknown project wasinject_colab_setup's raise — reached inside the per-script loop six lines later. Running it on an unregistered project deleted 113 tracked notebooks inautocti_workspaceand then aborted, leaving a stray intermediate.ipynbinsidescripts/. Onlygit checkoutrecovered it; in a repo with uncommitted notebook work, nothing would.The root-level
start_here*.pyloop (generate.py:83-91) does inject before the rmtree, so a workspace with a rootstart_here.pyfailed safely.autocti_workspacehas none. The safety was accidental, not designed — which is why this survived: the repos anyone normally runs this on all have a rootstart_here.py.Three separate sessions hit this independently before it got fixed.
The fix
generate_project_folders()and the rmtree, againstbuild_util.COLAB_PROJECTS— kept as the single source of truth rather than duplicating the project list.sys.exitmatches the existing guard style ingenerate_markdown.py:450.inject_colab_setup's own raise stays as the backstop for direct callers.finallyon the per-script loop that removes the intermediate notebook, so a mid-loop failure cannot strand an untracked.ipynbinsidescripts/.Verification
Tests are subprocess-driven:
generate.pyparses argv and does all its work at module level under__main__, so import cannot exercise it. The fixture is a realgit initworkspace with a trackednotebooks/tree.pytest tests/ -q→ 258 passed.git stash push -- autohands/generate.py), so they pin the bug rather than passing vacuously. The third — a known project is still accepted — passes either way by design, guarding against the check rejecting valid input.Live, on the real repos:
generate.py autoctifromautocti_workspace/scripts/dataset_1d/advanced/database/examples/data_fitting.ipynbgit statusempty, 79 notebooks intact, 0 straygenerate.py howtolensfromHowToLens/Deliberately out of scope
Registering
autoctifor Colab, the PyAutoNerves_PROJECTSentry,pre_build.shwiring, and regenerating autocti's 79 notebooks are owned byPyAutoMind/draft/release/autocti/cti_release_train_wiring.md(human-required), whose item 3 is exactly "generate.py autocti(blocked today by the COLAB_PROJECTS registry check) → commit regenerated notebooks", and whose item 2 carries the arcticpy recipe (aptlibgsl-dev,pip install numpy cython,arcticpy==2.6 --no-build-isolation --no-deps— a naive install downgrades numpy below 2.0).So
autocti_workspace/notebooks/keeps its 34Finish.cells and 4 mangled code cells until that task runs. It regenerates all 79 notebooks anyway, so clearing them here would be throwaway work — and declaring autocti "deliberately non-Colab" to unblock generation would contradict that task's intent. What changes today is that the blocker is now a clean refusal instead of a destructive one.🤖 Generated with Claude Code
https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM