Support string/byte arrays

[jakirkham]

Not sure if this is already supported and I'm just running into some kind of problem or if this is yet to be implemented. That said, I would really like to be able to store string and byte arrays. This would be of particular value when working with some kinds of large pieces of metadata.

[jakirkham]

FWIW I encountered some issue where JSON serializability came into question.

Edit: This was when in use with attrs. Though I have use cases for both attrs and arrays with string/byte data.

[alimanfoo]

Do you mean creating a zarr array of strings? Or storing strings in the attributes for an array or group? If it's the former, then there are a couple of options. One is to create a zarr array with S or U fixed size dtype. This should be supported currently. Another option is to create a zarr array with object dtype, which can handle variable length strings. To make that work you need to add a filter that knows how to encode/decode Python objects. There are two filters in numcodecs that can do this, one using pickle, the other using msgpack. Once I have numcodecs and zarr integrated, these should both be available. There have also been discussions of creating a codec for variable length strings that is cross-platform, don't have web to hand but there are some issues to xref.

On Mon, 27 Feb 2017 at 21:39, jakirkham ***@***.***> wrote: Not sure if this is already supported and I'm just running into some kind of problem or if this is yet to be implemented. That said, I would really like to be able to store string and byte arrays. This would be of particular value when working with some kinds of large pieces of metadata. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/alimanfoo/zarr/issues/135>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAq8Ql8iDt7z0PHiqLcf1lGlOOLlDaufks5rg0JzgaJpZM4MNqRM> .

-- Alistair Miles Head of Epidemiological Informatics Centre for Genomics and Global Health <http://cggh.org> The Wellcome Trust Centre for Human Genetics Roosevelt Drive Oxford OX3 7BN United Kingdom Email: alimanfoo@googlemail.com Web: http://purl.org/net/aliman Twitter: https://twitter.com/alimanfoo Tel: +44 (0)1865 287721