fix: stop silently passing any script with 'inversion' in its path - #225
Merged
Conversation
execute_script() caught CalledProcessError and, if the string 'inversion' appeared anywhere in the script path, recorded the run as PASSED with 'Inversion script failure (ignored)'. It was a substring match on the path, not on the exception, so a script could fail for any reason at all and still be reported green. Demonstrated on a file containing only : before: PASS (inversion) status=PASSED has_failures=False after: FAIL status=FAILED has_failures=True Three scripts workspace-wide match the substring, two of which are the chapter_4 tutorial_3_inversions files in HowToGalaxy and HowToLens — so the escape covered precisely the tutorial that shipped broken in HowToGalaxy #56/#57 and would have kept it green under any script run. The notebook runner never had this clause, which is why the notebook job reported that failure while a script run would not have. Its narrower cousin in execute_notebooks_in_folder (InversionException in the traceback) is left in place: it keys off the exception type rather than the filename, so it is deliberately out of scope here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
execute_script()caughtCalledProcessErrorand, if the stringinversionappeared anywhere in the script path, recorded the run asPASSEDwith"Inversion script failure (ignored)". It matched on the filename, not the exception — so a script could fail for any reason at all and still be reported green.Demonstrated on a file whose entire contents are
raise SystemExit("deliberate failure"), namedtutorial_3_inversions.py:has_failuresmainPASS (inversion, 0.0s)Status.PASSEDFalseFAIL (0.0s) deliberate failureStatus.FAILEDTrueWhy it matters
Three scripts workspace-wide match the substring — and two are the
chapter_4_pixelizations/tutorial_3_inversions.pyfiles in HowToGalaxy and HowToLens:So the escape covered precisely the tutorial that shipped broken in PyAutoLabs/HowToGalaxy#56 / #57, and would have kept it green under any script run.
execute_notebooks_in_foldernever had this clause, which is exactly why the notebook job reported that failure while a script run never would have.This blocks PyAutoLabs/HowToGalaxy#58, which expands HowTo smoke to run every script: without this change the newly-covered
tutorial_3_inversions.pywould be unconditionally green, defeating the point of the coverage work.Deliberately left in place
execute_notebooks_in_folderhas a narrower cousin —if "InversionException" in traceback.format_exc()→ PASS. It keys off the exception type rather than the filename, so it is defensible for genuinely data-dependent inversion failures and is out of scope here. Flagging it for a separate decision. The adjacentis_clean_skip_exit()pass is correct as-is (asys.exit(0)skip guard is a clean exit by design).Test Plan
Generated by the PyAutoLabs agent workflow.