Skip to content

Existing v2 big-endian arrays cannot be read #59

@melissalinkert

Description

@melissalinkert

As a minimal test, I created a small v2 array using zarr-python (2.8.1):

import zarr
arr = zarr.open('v2-big-endian.zarr', mode='w', shape=(1, 1), chunks=(1, 1), dtype='>u2')
arr[:] = 65280

This can be re-opened by zarr-python, with the expected array contents of 65280. The created zarr is v2-big-endian.tar.gz.

If I then try to read the array with zarr-java 0.0.10:

$ cat Test.java 
import dev.zarr.zarrjava.store.FilesystemStore;
import dev.zarr.zarrjava.v2.Array;

public class Test {
  public static void main(String[] args) throws Exception {
    FilesystemStore store = new FilesystemStore(args[0]);
    Array arr = Array.open(store.resolve());
  }
}
$ java Test v2-big-endian.zarr/
Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `dev.zarr.zarrjava.v2.DataType` from String ">u2": not one of the values accepted for Enum class: [|i1, <i2, <i4, |u1, <u2, <f4, |b1, <f8, <u8, <u4, <i8]
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 13, column: 14] (through reference chain: dev.zarr.zarrjava.v2.ArrayMetadata["dtype"])
	at com.fasterxml.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67)
	at com.fasterxml.jackson.databind.DeserializationContext.weirdStringException(DeserializationContext.java:2014)
	at com.fasterxml.jackson.databind.DeserializationContext.handleWeirdStringValue(DeserializationContext.java:1300)
	at com.fasterxml.jackson.databind.deser.std.EnumDeserializer._deserializeAltString(EnumDeserializer.java:454)
	at com.fasterxml.jackson.databind.deser.std.EnumDeserializer._fromString(EnumDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.std.EnumDeserializer.deserialize(EnumDeserializer.java:275)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:553)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:597)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:440)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1499)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:340)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:5038)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:4015)
	at dev.zarr.zarrjava.v2.Array.open(Array.java:54)
	at Test.main(Test.java:7)

Creating a test zarr that mimics real-world data with bioformats2raw 0.12.0-rc1 and trying to read that results in a similar error:

$ bin/bioformats2raw "test&pixelType=uint16&little=false.fake" v2-bioformats2raw.zarr
$ java Test v2-bioformats2raw.zarr/0/0
Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `dev.zarr.zarrjava.v2.DataType` from String ">u2": not one of the values accepted for Enum class: [|i1, <i2, <i4, |u1, <u2, <f4, |b1, <f8, <u8, <u4, <i8]
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 10, column: 13] (through reference chain: dev.zarr.zarrjava.v2.ArrayMetadata["dtype"])
	at com.fasterxml.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67)
	at com.fasterxml.jackson.databind.DeserializationContext.weirdStringException(DeserializationContext.java:2014)
	at com.fasterxml.jackson.databind.DeserializationContext.handleWeirdStringValue(DeserializationContext.java:1300)
	at com.fasterxml.jackson.databind.deser.std.EnumDeserializer._deserializeAltString(EnumDeserializer.java:454)
	at com.fasterxml.jackson.databind.deser.std.EnumDeserializer._fromString(EnumDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.std.EnumDeserializer.deserialize(EnumDeserializer.java:275)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:553)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:597)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:440)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1499)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:340)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:5038)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:4015)
	at dev.zarr.zarrjava.v2.Array.open(Array.java:54)
	at Test.main(Test.java:7)

The generated test zarr is v2-bioformats2raw.tar.gz.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions