Skip to content

feat(check_dataset_allowlist): fail when a capped should_simulate would delete committed data #252

Description

@Jammy2211

Follow-up to PyAutoArray#470 / autolens_workspace_test#264. That PR fixed the single live instance by declaring full_datasets on the one offending script; this closes the general footgun and corrects a docstring that actively misleads about it.

Part 1 — extend the allowlist guard (PyAutoHands)

autohands/check_dataset_allowlist.py asserts that every tracked file under dataset/ is covered by the workspace's !dataset/... allowlist — i.e. that nothing generated got committed. It does not check the mirror-image failure: that nothing committed gets deleted.

should_simulate() ends in shutil.rmtree. A script that reaches a committed dataset while PYAUTO_SMALL_DATASETS=1 is still in force destroys data the allowlist exists to protect. The library-side stamp guard (_is_capped_at_the_current_cap) cannot cover this in general — it reads <dataset>/data.fits, so any JSON-only dataset is invisible to it.

Add that second leg. Releasing tokens are derived from env_config.ENV_DECLARATION_TOKENS rather than hardcoded, so a future token that releases the var is picked up automatically.

Part 2 — correct the should_simulate docstring (PyAutoArray)

Its "Known gap" section ends:

Note that point-source datasets are not in this gap: they write a top-level data.fits alongside their JSON and are covered normally.

True in autolens_workspace. False in autolens_workspace_test, whose dataset/point_source/simple is three tracked JSON files and no FITS — that repo's .gitignore lists data.fits under "Generated artifacts — never check in". This is precisely the directory #470 was about, so the docstring currently reassures the reader about the one case that bit us.

Docstring-only; no behaviour change.

Design constraint

Matching a call site to a dataset directory is the sharp edge. This gate runs in pre_build, where a false positive blocks a release, so:

  • resolution is exact (restricted AST evaluator), never fuzzy;
  • anything unresolvable is reported and skipped, never guessed;
  • the skipped count is always printed, so a partial sweep cannot read as full coverage.

Under-reporting is acceptable and visible. Over-reporting is not.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions