Add OBN Continuous Receiver Gathers (CRG) template - #850
Draft
andgineer wants to merge 3 commits into
Draft
Conversation
The inserted `trace` dimension was hardcoded to chunk size 1 and an int16 counter, which caps a gather at ~32k duplicates and forces one chunk per trace. Continuous receiver gathers need both knobs: ~2.6e5 segments per receiver, and a chunk size large enough to reach a sane object size. `GridOverrides.chunksize` now applies to the `has_duplicates` path as well as `non_binned`, and a new optional `GridOverrides.trace_dtype` selects the counter dtype. Both default to the previous behaviour (chunk 1, int16), so existing payloads that omit them are unaffected. Also generalizes `_resolve_synthesize_dims` to read the template's `synthesize_missing_dims` attribute instead of isinstance-checking the OBN template, so any template can declare dimensions it synthesizes when they are absent from the SEG-Y spec. The ingestion validator keys off the same attribute. Co-authored-by: Cursor <cursoragent@cursor.com>
Continuous receiver gathers record every receiver continuously rather than per shot, so there is no shot index to grid on: the natural key is (component, receiver_line, receiver), and each receiver holds a long run of time segments that share that key. `ObnContinuousReceiverGathers3D` grids on those three spatial dimensions and leaves the segment axis to the `HasDuplicates` override, which inserts `trace` ahead of the vertical axis. The resulting layout is (component, receiver_line, receiver, trace, time). `component` is declared in `synthesize_missing_dims`, so single-component files that carry no component header ingest against the same template with the dimension synthesized. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — opening early to start the discussion on the two API questions at the bottom. Code and tests are complete and green locally; happy to reshape the public surface before this is reviewed properly.
What this adds
A template for OBN continuous receiver gathers (CRG). Unlike shot-organised OBN data, CRG receivers record continuously, so there is no shot index to grid on. The natural key is
(component, receiver_line, receiver), and each receiver holds a long run of time segments sharing that key.Rather than introduce a new grid-override mechanism, this extends the existing
HasDuplicatespath, which already solves exactly this problem — it appends a per-tuple counter as a synthetictracedimension. Two knobs were missing for the CRG scale, so the first commit adds them and the second adds the template.Resulting layout:
(component, receiver_line, receiver, trace, time).Commits
1. Expose chunk size and counter dtype on the
HasDuplicatespathThe inserted
tracedimension was hardcoded to chunk size 1 and anint16counter. That caps a gather at ~32k duplicates and forces one chunk per trace. A full CRG campaign runs to roughly 2.6e5 segments per receiver, and chunk-per-trace gives unusably small objects.GridOverrides.chunksizenow applies tohas_duplicatesas well asnon_binned.GridOverrides.trace_dtypeselects the counter dtype.Both default to the current behaviour (chunk 1,
int16), so payloads that omit them are byte-identical to today. There is an explicit backwards-compatibility test for that.The same commit generalizes
_resolve_synthesize_dimsto read a template'ssynthesize_missing_dimsattribute instead of isinstance-checkingSeismic3DObnReceiverGathersTemplate. The OBN template already sets that attribute, so its behaviour is unchanged; this just lets other templates opt in.ingestion/segy/validation.pykeys its optional-dimension exemption off the same attribute so the two cannot drift.2. Add OBN Continuous Receiver Gathers (CRG) template
ObnContinuousReceiverGathers3D, registered as a built-in. Grids on the three spatial dimensions and leaves the segment axis toHasDuplicates.componentis declared insynthesize_missing_dims, so single-component files carrying no component header ingest against the same template with the dimension synthesized — one template covers both layouts.Tests
tests/unit tests/integration -m "not cloud"→ 560 passed.tracehonourschunksizeandtrace_dtype, plus a BC test asserting the default path still yields chunk 1 /int16.componentpath.Questions for maintainers
Is the API shape right for the two new knobs? I put
trace_dtypeonGridOverridesalongside the existingchunksize, since that is wherenon_binnedalready carries its chunk size and it keeps the two paths symmetric. The alternative is to carry them on the template instead, which would keepGridOverridesnarrower but split the configuration for one dimension across two objects. Happy to move it.Is "extend
HasDuplicates" the direction you want? The alternative is a dedicated override (something likeCalculateTimeSegmentIndex) that does not touch the existing path at all. Extending seemed better because the semantics are genuinely identical — a counter disambiguating repeated index tuples — and a separate override would duplicate the machinery. But it does widen a shipping public surface, so say the word if you would rather have this isolated.