fix: allow require_array to accept a ZDType - #4189
Conversation
AsyncGroup.require_array normalised its dtype with np.dtype(), which cannot consume a ZDType, so requiring an existing array with one raised a TypeError. Every sibling creation method already accepts ZDTypeLike. Widen the annotation and normalise via parse_data_type().to_native_dtype(). parse_data_type(None) resolves to float64 just as np.dtype(None) did, so the default is unchanged. This leaves numpy.typing unused, so drop it.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4189 +/- ##
=======================================
Coverage 94.00% 94.00%
=======================================
Files 91 91
Lines 12791 12791
=======================================
Hits 12024 12024
Misses 767 767
🚀 New features to boost your workflow:
|
parse_data_type does not accept None, so pass "float64" directly, which is what np.dtype(None) resolved to before.
| await root.require_array("bar", shape=(10,), dtype="int8") | ||
|
|
||
|
|
||
| async def test_require_array_zdtype(store: Store, zarr_format: ZarrFormat) -> None: |
There was a problem hiding this comment.
can you parametrize this test over (input, expected output) pairs, and ensure that the dtype=None case is included? Pairs like [("int32", Int32()), (None, Int64()]
There was a problem hiding this comment.
correction: the pair for the None case would be (None, Float64())
|
hi @JOhnsonKC201 is it OK if I push some changes to your branch to get some of the remaining issues resolved? we just need a few changes before this can be merged |
Covers the `dtype=None` path, which resolves to float64 and was previously untested, and asserts on the resulting ZDType rather than the native dtype.
|
Sure, go ahead. I just pushed 98376b1, which parametrizes Happy for you to push whatever else is needed to get this over the line. |
45a29dd to
f1fd70a
Compare
|
thanks @JOhnsonKC201! |
Resolves an import-block conflict in `src/zarr/core/group.py`: zarr-developers#4189 added `parse_data_type` at the same spot this branch added `parse_field`. Both imports are kept. Assisted-by: ClaudeCode:claude-opus-5
Summary
Closes #3377.
Group.require_arraycould not accept a Zarr data type, even though every otherarray-creation method on the group already does. Requiring an array that exists
raised:
AsyncGroup.require_arraywas annotatednpt.DTypeLike | Noneand normalisedits argument with
np.dtype(dtype), which cannot consume aZDType. Itssiblings (
create_arrayand friends) takeZDTypeLike. So the "array alreadyexists" branch always blew up for a
ZDType, while the string and NumPy formsworked.
The fix widens the annotation to
ZDTypeLike | Noneand normalises withparse_data_type(...).to_native_dtype().AsyncArray.dtypealready returns thenative dtype, so the
exact/np.can_castcomparisons below are unchanged.Two follow-on details:
parse_data_type(None)resolves tofloat64, which isexactly what
np.dtype(None)gave, so the default path is unchanged; andremoving the last real use of
numpy.typingleft the import unused, so it isdropped (the three remaining
npt.mentions in this file are in docstrings).For reviewers
The bit worth a second look is the
Nonecase, sincenp.dtype(None)silentlymeans
float64and I wanted to preserve that rather than start rejecting it. Ichecked all four input forms round-trip to the same array dtype:
ZDType,np.dtype,"int32", andNone.I am confident about the rest: it is a normalisation swap, and behavior for
string and NumPy dtypes is identical.
Author attestation
TODO
docs/user-guide/*.mdchanges/Test plan
tests/test_group.py::test_require_array_zdtype, coveringZDType,np.dtypeandstr. It fails on main with theTypeErrorabove.tests/test_group.py: 529 passed, 15 skipped.tests/test_api.py tests/test_array.py: 1654 passed, 44 skipped.ruff checkandruff format --checkclean.