Conversation
Contributor
|
i can understand how some folks might not like the churn but I'm +1 on this. we can always go back to hatch if we don't like plain uv. |
Contributor
|
@maxrjones did you consider putting some of these scripts into a runner like a justfile? |
Member
Author
I considered nox/tox, but decided those were overkill after a chat with @chuckwondo. Just might be a better fit |
Contributor
|
i mention |
maxrjones
commented
Jun 24, 2026
| 'packaging>=22.0', | ||
| 'numpy>=2', | ||
| 'numcodecs>=0.14', | ||
| 'numcodecs>=0.14.1', |
Member
Author
There was a problem hiding this comment.
Zarr-Python was actually not compatible with numcodecs 0.14.0; the lowest supported version is 0.14.1 in reality due to the need for the soft dependency fixes in https://github.com/zarr-developers/numcodecs/releases/tag/v0.14.1.
Contributor
|
At first glance I'm opposed to it because now we're maintaining a script. There's also now a cool vscode plugin for hatch environments: https://github.com/pypa/hatch-code I will set this to unread and look later though |
Contributor
|
would it make sense to keep the justfile and defer the "drop or keep hatch" decision? I think a single file for all the verbs is a nice idea on its own |
4 tasks
d-v-b
added a commit
to d-v-b/zarr-python
that referenced
this pull request
Sep 17, 2026
… path Review follow-ups to the Justfile centralization: - `just gpu` read HATCH_ENV, so an exported CPU test environment silently redirected `pytest -m gpu` into an env built without the gpu feature. It now reads GPU_HATCH_ENV, which nothing else sets. - Pin `uvx prek` to 0.5.3 and restore hook-environment caching in the lint workflow, so lint no longer floats on whatever prek PyPI serves that day. - Restore the mypy hook's `uv run --frozen mypy` entry: routing it through `just typecheck` made just a prerequisite for committing in every existing clone, and expanded to the identical command. - Run the changelog filename check with `uv run --no-project python` instead of building the `dev` hatch env (test + remote-tests + docs + mypy) for a script that imports only sys and pathlib. - Extend `just just-check` over packages/*/justfile, which the root delegation recipes depend on and the formatting gate did not cover. - Set up Python in the codspeed job before pip-installing into it, matching the other workflows; that runner image was never given one. - Revert the release and nightly-wheel jobs to `hatch build`, so the publishing path does not fetch rust-just to run a pure alias. - Renumber the changelog fragment to this pull request and credit zarr-developers#4096 in the body, so the rendered note does not link to an unrelated upstream PR. Assisted-by: ClaudeCode:claude-opus-5
d-v-b
added a commit
that referenced
this pull request
Sep 20, 2026
* chore: extract just recipes from hatch replacement proposal Extracts the Justfile from 98dcde7 (#4096). Environment and CI migration changes are adapted separately. Assisted-by: Codex:GPT-6 * chore: use just for python helpers Extracts the Justfile changes from 693f694 (#4096). Assisted-by: Codex:GPT-6 * chore: centralize development and CI verbs in just Keep Hatch environment definitions while moving task commands into Justfile. Route root CI, Read the Docs, and contributor workflows through the same recipes, preserve quoted arguments, and ship recipes and helpers in the sdist. Assisted-by: Codex:GPT-6 * chore: delegate package commands to their justfiles Expose package recipes from the root while retaining each package’s command definitions and working directory. Assisted-by: Codex:GPT-6 * fix(ci): harden the just migration's environment, pinning and release path Review follow-ups to the Justfile centralization: - `just gpu` read HATCH_ENV, so an exported CPU test environment silently redirected `pytest -m gpu` into an env built without the gpu feature. It now reads GPU_HATCH_ENV, which nothing else sets. - Pin `uvx prek` to 0.5.3 and restore hook-environment caching in the lint workflow, so lint no longer floats on whatever prek PyPI serves that day. - Restore the mypy hook's `uv run --frozen mypy` entry: routing it through `just typecheck` made just a prerequisite for committing in every existing clone, and expanded to the identical command. - Run the changelog filename check with `uv run --no-project python` instead of building the `dev` hatch env (test + remote-tests + docs + mypy) for a script that imports only sys and pathlib. - Extend `just just-check` over packages/*/justfile, which the root delegation recipes depend on and the formatting gate did not cover. - Set up Python in the codspeed job before pip-installing into it, matching the other workflows; that runner image was never given one. - Revert the release and nightly-wheel jobs to `hatch build`, so the publishing path does not fetch rust-just to run a pure alias. - Renumber the changelog fragment to this pull request and credit #4096 in the body, so the rendered note does not link to an unrelated upstream PR. Assisted-by: ClaudeCode:claude-opus-5 * fix(ci): install prek persistently for git hooks; run setup steps in one just process `just hooks-install` ran `uvx prek install`, and prek writes a hook shim that hard-codes the binary it was installed from, falling back to `prek` on PATH. Under uvx that path is an entry in uv's archive cache, so the first `uv cache prune` (or a bump of the pinned version) broke every commit with `prek: not found`, and nothing put `prek` on PATH for the fallback. Install it as a pinned `uv tool` instead: a changed pin upgrades in place, a repeated run is a no-op, and the shim points at a path that stays. `setup` re-invoked `just list-env`, which dropped a `just hatch_env=... setup` override on the floor because the child process re-read HATCH_ENV. Run it as a subsequent dependency so both steps see the same value. Also retire the last two comments naming the removed hatch scripts. Assisted-by: ClaudeCode:claude-fable-5-1 * chore: renumber the changelog fragment to the upstream pull request towncrier renders the fragment name as a link into zarr-developers/zarr-python, so the fork's PR number would have pointed the released note at an unrelated upstream pull request. #4372 is the PR that merges this work. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: install just as a binary in the benchmark job Adding `actions/setup-python` to this job so that `pip install rust-just` had an interpreter also changed which interpreter hatch built `test.py3.12-minimal` on: the runner's own Python was replaced by /opt/hostedtoolcache/Python/3.12.5/arm64. CodSpeed measured the result as a 12.97% degradation across 26 benchmarks, spread -12.76% to -13.99% -- a range of 1.2 points over slice indexing, sharded Morton indexing, three array shapes and two store types. Nothing in this branch touches src/, tests/benchmarks/ or packages/; that uniformity is an interpreter swap, not a regression, and CodSpeed flagged it as "different runtime environments detected". Install just with extractions/setup-just instead, matching the three package workflows. No interpreter is set up, so hatch resolves the same Python the baseline used, and `just-version` is pinned to satisfy zizmor's unpinned-tools audit. The benchmarks need a re-run to produce a comparison against a matching environment; the numbers on the previous run should not be acknowledged as a regression. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: install just from official release binaries; lock the docs toolchain Follow-ups from an adversarial audit of this branch. Supply chain. `rust-just` is a third-party repackaging of just on PyPI (Repository: github.com/gnpaone/rust-just) with no attestation on any of its 17 wheels, and this branch had introduced it in ten places including the self-hosted GPU runner and Read the Docs. All 23 just installs across the repo now come from casey/just's own release binaries via extractions/setup-crate@7577c1bd (v2.0.1), a node action whose SHA pins every line of executed code -- rather than extractions/setup-just, a composite wrapper that astral-sh/python-build-standalone#771 moved off for that reason. This also retires the 12 pre-existing setup-just call sites, so the repo has one mechanism instead of four. setup-crate does not verify checksums (extractions/setup-just#20 is open); Read the Docs, where no action is available, fetches the release tarball and verifies it against the SHA256SUMS casey/just publishes. Lockfile. docs.yml had stopped resolving from uv.lock: `uv sync --group docs` became `hatch run docs:`, and hatch never reads the lockfile, so the entire docs toolchain floated -- including numcodecs[msgpack], which carries no version constraint. Dependabot's uv ecosystem exists to keep that lock fresh and no job exercised it any more. The eight docs and changelog recipes now run `uv run --frozen --group docs`, so single-version tooling comes from the lock (as mypy already did) while hatch keeps the test environments, which exist per interpreter and per dependency set and cannot live in one lockfile. The hatch docs environment is removed as unused, and neither docs.yml nor Read the Docs installs hatch at all now. Argument forwarding. The three package justfiles splatted `{{ args }}` unquoted, so `just zarr-metadata test -k 'a and b'` reached pytest as `-k a` plus two stray paths, and command substitution in an argument executed. They now use `set positional-arguments` and `"$@"` like the root, which is what the contributing guide already claimed. Also, regressions from my own earlier commits on this branch: `just lint` had dropped `--show-diff-on-failure --color=always`, which j178/prek-action passed by default, so a hook that rewrote a file failed with no diff; a four-line comment above `hooks-install` displaced its `just --list` description, since just reads only the last comment line; the prek cache key did not cover prek_version and had a restore-keys fallback that could reuse a store built by a different prek; `just just-check` ran before `just lint`, so a cosmetic formatting nit hid every real lint result; and the `packages/*/justfile` glob lacked nullglob, which breaks in the sdist, where /Justfile ships but /packages deliberately does not. Docs: the install instructions used `pip install`, which fails on an externally-managed interpreter; `just just-check` was an undocumented required check; two sentences still told contributors to activate an environment that no longer exists; and the changelog fragment did not mention that the Hatch script tables were removed. Adds a .gitattributes rule so a Windows checkout does not get a CRLF Justfile that `just --fmt --check` rejects on every line. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR switches to using uv + just + hatchling rather than uv + hatch + hatchling.
For context, I think it's sub-optimal to have environments managed by both hatch (for CI and in contributing guide) and uv (for typing). This PR shows how we could use only uv via a small set of bash scripts that make up for the missing task runner and environment names features in uv relative to hatch. The benefit is that all of our CI and developer workflows would be based on the exact same lock files for improved reproducibility.
I tried hatch's new lockfile support first, but it's platform specific so it doesn't help us much given we want cross-platform consistency.
I don't feel strongly about this PR, so totally fine being told that we should stick with the status quo.
TODO:
docs/user-guide/*.mdchanges/