Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make readers pluggable via entrypoint system #245

Open
TomNicholas opened this issue Oct 3, 2024 · 0 comments · Fixed by #261
Open

Make readers pluggable via entrypoint system #245

TomNicholas opened this issue Oct 3, 2024 · 0 comments · Fixed by #261
Labels
enhancement New feature or request references generation Reading byte ranges from archival files

Comments

@TomNicholas
Copy link
Member

We could make an entrypoint system for virtualizarr, in addition to plugging in to xarray's backend entrypoint system. That would allow this syntax for creating virtual xr.Dataset objects containing ManifestArrays, e.g.

vds = vz.open_virtual_dataset(grib_paths, reader='hypergrib')

(as suggested in #238)

This would allow users to freely create their own virtual readers / metadata parsers (whatever we call them #239) to deal with their own gnarly file formats, without requiring upstream contributions. This model for extendability has worked very well for xarray, and it's completely analogous to the problem we have here.

If we combined this with #35, we would then have a virtualizarr xarray backend entrypoint, which calls out to specific virtualizarr readers via another entrypoint. That would allow this syntax.

vds = xr.open_dataset(grib_paths, engine='virtualizarr', reader='hypergrib')

I think we should make the entrypoint system in virtualizarr first before we decide if entrypoints calling entrypoints is actually a good idea... 🐶

cc @ayushnag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request references generation Reading byte ranges from archival files
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant