Skip to content

feat: url-pipeline core — parser, adapter ABC, registry, store hooks - #6

Closed
jhamman wants to merge 2 commits into
feature/zipstore-file-likefrom
feature/url-pipeline-core
Closed

feat: url-pipeline core — parser, adapter ABC, registry, store hooks#6
jhamman wants to merge 2 commits into
feature/zipstore-file-likefrom
feature/url-pipeline-core

Conversation

@jhamman

@jhamman jhamman commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Summary

Implements URL pipeline support (https://github.com/jbms/url-pipeline): '|'-chained URLs resolve through pluggable adapters registered under the 'zarr.url_adapters' entry-point group (entry-point name = URL scheme).

  • zarr.abc.url_pipeline: PipelineSegment, AdapterResolution, PipelineContext, URLPipelineAdapter (single-classmethod contract)
  • zarr.storage._url_pipeline: parse_pipeline / resolve_pipeline; the root sub-URL delegates to make_store so existing file/memory/fsspec routing is unchanged
  • registry: register_url_adapter / get_url_adapter / list_url_adapter_schemes (name check only; no adapter imports)
  • make_store/make_store_path route strings containing '|' (or a registered root scheme) through the resolver; residual store paths combine with the user-supplied path
  • StorePath gains a zarr_format attribute (populated by format segments in a follow-up)

For reviewers

This is the second of a series of Pull Requests toward zarr-developers#2943.

Author attestation

  • I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/user-guide/*.md
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@jhamman
jhamman force-pushed the feature/url-pipeline-core branch 2 times, most recently from 03bed15 to 5ae451e Compare July 27, 2026 02:12
@jhamman
jhamman force-pushed the feature/zipstore-file-like branch from 7d33da8 to ef00953 Compare July 27, 2026 15:12
@jhamman
jhamman force-pushed the feature/url-pipeline-core branch 2 times, most recently from fb71546 to ec2c8bf Compare July 27, 2026 19:10
@jhamman
jhamman force-pushed the feature/zipstore-file-like branch from 54fa1dd to b5f99fb Compare July 27, 2026 19:14
@jhamman
jhamman force-pushed the feature/url-pipeline-core branch from ec2c8bf to f26b436 Compare July 27, 2026 19:14
…#4187)

Allows constructing a ZipStore from any seekable binary reader, enabling
zip archives on remote storage:

- io objects (BytesIO, fsspec file objects) are used directly
- minimal readers that are not io.IOBase instances and whose read() may
  return buffer-protocol objects rather than bytes (e.g.
  obstore.ReadableFile) are adapted via a small io.RawIOBase wrapper
  when opened for reading

clear()/move() raise NotImplementedError for file-object-backed stores.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@jhamman
jhamman force-pushed the feature/url-pipeline-core branch from f26b436 to d77e88f Compare July 28, 2026 02:33
Implements URL pipeline support (https://github.com/jbms/url-pipeline):
'|'-chained URLs resolve through pluggable adapters registered under the
'zarr.url_adapters' entry-point group (entry-point name = URL scheme).

- zarr.abc.url_pipeline: PipelineSegment, AdapterResolution,
  PipelineContext, URLPipelineAdapter (single-classmethod contract)
- zarr.storage._url_pipeline: parse_pipeline / resolve_pipeline; the root
  sub-URL delegates to make_store so existing file/memory/fsspec routing
  is unchanged
- registry: register_url_adapter / get_url_adapter /
  list_url_adapter_schemes (name check only; no adapter imports)
- make_store/make_store_path route strings containing '|' (or a
  registered root scheme) through the resolver; residual store paths
  combine with the user-supplied path
- StorePath gains a zarr_format attribute (populated by format segments
  in a follow-up)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jhamman
jhamman force-pushed the feature/url-pipeline-core branch from d77e88f to 10cb73b Compare July 28, 2026 12:30
@jhamman jhamman closed this Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant