[Feature]: Reference Updates

[mavaylon1]

What would you like to see added to HDMF-ZARR?

Currently, HDMF-Zarr is behind HDMF in how it handles export and writing references (via a queue). Without a queue, there is no guarantee that the references will be resolved as the order of how objects are written is not set. The queue will set aside the references until everything has been written, after which the references will be resolved and written.

Is your feature request related to a problem?

No response

What solution would you like?

Update the backend to current HDMF format.

Do you have any interest in helping implement the feature?

Yes.

Code of Conduct

• I agree to follow this project's Code of Conduct
• Have you checked the Contributing document?
• Have you ensured this change was not already requested?

[oruebel]

Just for additional detail. I believe in HDF5 the queue is essential, because references to objects cannot be generated until the object that is being referenced exists. Since ZarrIO stores references as a JSON with the paths to the objects, this may particular aspect may not necessarlily be an issues as long as we can: a) determine the path the object/builder will have and b) ensure that the object/builder will be saved eventually at that location.

That being said, the queue approach has several advantages:

• ensures that objects that are being linked to exist before creating links to them
• likely facilitates error checking when writing references
• could potentially help with creating references during export (but I'm not sure on this on)

[mavaylon1]

We have decided that with zarr we do not need to have the same queue system.