Add comparison for Xarray folks#1
Merged
Merged
Conversation
Contributor
Author
|
Was just exploring another avenue - what if we used a different backend (i.e., "cubed" instead of "dask") to see what that performance looks like. I ran into this error out of the box python xarray-profile.py
Traceback (most recent call last):
...
File "/Users/Hodgs004/coding/repos/xarray-interpolation/xarray-profile.py", line 187, in run
ds.isel(positions).compute()
~~~~~~~^^^^^^^^^^^
File "/Users/Hodgs004/coding/repos/xarray-interpolation/.pixi/envs/default/lib/python3.14/site-packages/xarray/core/dataset.py", line 2867, in isel
return self._isel_fancy(indexers, drop=drop, missing_dims=missing_dims)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/Hodgs004/coding/repos/xarray-interpolation/.pixi/envs/default/lib/python3.14/site-packages/xarray/core/dataset.py", line 2923, in _isel_fancy
new_var = var.isel(indexers=var_indexers)
File "/Users/Hodgs004/coding/repos/xarray-interpolation/.pixi/envs/default/lib/python3.14/site-packages/xarray/core/variable.py", line 1141, in isel
return self[key]
~~~~^^^^^
File "/Users/Hodgs004/coding/repos/xarray-interpolation/.pixi/envs/default/lib/python3.14/site-packages/xarray/core/variable.py", line 831, in __getitem__
data = indexing.apply_indexer(indexable, indexer)
File "/Users/Hodgs004/coding/repos/xarray-interpolation/.pixi/envs/default/lib/python3.14/site-packages/xarray/core/indexing.py", line 1182, in apply_indexer
return indexable.vindex[indexer]
~~~~~~~~~~~~~~~~^^^^^^^^^
File "/Users/Hodgs004/coding/repos/xarray-interpolation/.pixi/envs/default/lib/python3.14/site-packages/xarray/core/indexing.py", line 467, in __getitem__
return self.getter(key)
~~~~~~~~~~~^^^^^
File "/Users/Hodgs004/coding/repos/xarray-interpolation/.pixi/envs/default/lib/python3.14/site-packages/xarray/core/indexing.py", line 1783, in _vindex_get
raise TypeError("Vectorized indexing is not supported")
TypeError: Vectorized indexing is not supporteddue to these lines https://github.com/pydata/xarray/blob/a1408234d1dc786965bfb6311f8782561ee0e676/xarray/core/indexing.py#L1782-L1783 . Seems that vectorized indexing is relatively new to the array API (data-apis/array-api#669 , data-apis/array-api#900) but there are still some limitations that (I think) prevent us from doing vectorized indexing - hopefully Stephan or someone can shed more light ehre. Anyway - extracting the cubed array, trying the following (as expected) doesn't work either: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding a comparison here to discuss with the Xarray folks at some point. See README
Keen to chat about this @fluidnumericsJoe in our meeting today
cc @erikvansebille