Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ArrayMetadata{V3,V2} constructors #97

Merged
merged 1 commit into from
Nov 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions zarrs/src/array/array_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -314,18 +314,20 @@ impl ArrayBuilder {
self.bytes_to_bytes_codecs.clone(),
);

let array_metadata = ArrayMetadata::V3(ArrayMetadataV3::new(
self.shape.clone(),
self.data_type.metadata(),
self.chunk_grid.create_metadata(),
self.chunk_key_encoding.create_metadata(),
self.data_type.metadata_fill_value(&self.fill_value),
codec_chain.create_metadatas(),
self.attributes.clone(),
self.storage_transformers.create_metadatas(),
self.dimension_names.clone(),
self.additional_fields.clone(),
));
let array_metadata = ArrayMetadata::V3(
ArrayMetadataV3::new(
self.shape.clone(),
self.chunk_grid.create_metadata(),
self.data_type.metadata(),
self.data_type.metadata_fill_value(&self.fill_value),
codec_chain.create_metadatas(),
)
.with_attributes(self.attributes.clone())
.with_additional_fields(self.additional_fields.clone())
.with_chunk_key_encoding(self.chunk_key_encoding.create_metadata())
.with_dimension_names(self.dimension_names.clone())
.with_storage_transformers(self.storage_transformers.create_metadatas()),
);

Ok(Array {
storage,
Expand Down
14 changes: 10 additions & 4 deletions zarrs_metadata/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- Add `GroupMetadataV2` constructors
- Add `ArrayMetadataV2` constructors

### Changed
- **Breaking**: Mark `GroupMetadataV3` and `ArrayMetadataV3` as non-exhaustive
- **Breaking**: Bump MSRV to 1.77 (21 March, 2024)
- Refactor `GroupMetadata{V3,V2}` constructors
- **Breaking**: `GroupMetadataV3::new()` is now parameter free
- Add `GroupMetadataV2::new()`
- Add `GroupMetadata{V3,V2}::with_{attributes,additional_fields}()`
- Refactor `GroupMetadataV3` constructors
- **Breaking**: `GroupMetadataV3::new()` is now parameter free in favor of `with_` methods
- Add `GroupMetadataV3::with_{attributes,additional_fields}()`
- Refactor `ArrayMetadataV3` constructors
- **Breaking**: `ArrayMetadataV3::new()` takes fewer parameters in favor of `with_` methods
- Add `ArrayMetadataV3::with_{attributes,additional_fields,chunk_key_encoding,dimension_names,storage_transformers}`

## [0.1.0] - 2024-09-02

Expand Down
63 changes: 63 additions & 0 deletions zarrs_metadata/src/v2/array.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,69 @@
pub additional_fields: AdditionalFields,
}

impl ArrayMetadataV2 {
/// Create Zarr V2 array metadata.
///
/// Defaults to:
/// - C order,
/// - empty attributes, and
/// - no additional fields.
#[must_use]
pub fn new(
shape: ArrayShape,
chunks: ChunkShape,
dtype: DataTypeMetadataV2,
fill_value: FillValueMetadataV2,
compressor: Option<MetadataV2>,
filters: Option<Vec<MetadataV2>>,
) -> Self {
Self {
zarr_format: monostate::MustBe!(2u64),
shape,
chunks,
dtype,
compressor,
fill_value,
order: ArrayMetadataV2Order::C,
filters,
dimension_separator: ChunkKeySeparator::Dot,
attributes: serde_json::Map::default(),
additional_fields: AdditionalFields::default(),
}
}

Check warning on line 116 in zarrs_metadata/src/v2/array.rs

View check run for this annotation

Codecov / codecov/patch

zarrs_metadata/src/v2/array.rs#L95-L116

Added lines #L95 - L116 were not covered by tests

/// Set the dimension separator.
#[must_use]
pub fn with_dimension_separator(mut self, dimension_separator: ChunkKeySeparator) -> Self {
self.dimension_separator = dimension_separator;
self
}

Check warning on line 123 in zarrs_metadata/src/v2/array.rs

View check run for this annotation

Codecov / codecov/patch

zarrs_metadata/src/v2/array.rs#L120-L123

Added lines #L120 - L123 were not covered by tests

/// Set the order.
#[must_use]
pub fn with_order(mut self, order: ArrayMetadataV2Order) -> Self {
self.order = order;
self
}

Check warning on line 130 in zarrs_metadata/src/v2/array.rs

View check run for this annotation

Codecov / codecov/patch

zarrs_metadata/src/v2/array.rs#L127-L130

Added lines #L127 - L130 were not covered by tests

/// Set the user attributes.
#[must_use]
pub fn with_attributes(
mut self,
attributes: serde_json::Map<String, serde_json::Value>,
) -> Self {
self.attributes = attributes;
self
}

Check warning on line 140 in zarrs_metadata/src/v2/array.rs

View check run for this annotation

Codecov / codecov/patch

zarrs_metadata/src/v2/array.rs#L134-L140

Added lines #L134 - L140 were not covered by tests

/// Set the additional fields.
#[must_use]
pub fn with_additional_fields(mut self, additional_fields: AdditionalFields) -> Self {
self.additional_fields = additional_fields;
self
}

Check warning on line 147 in zarrs_metadata/src/v2/array.rs

View check run for this annotation

Codecov / codecov/patch

zarrs_metadata/src/v2/array.rs#L144-L147

Added lines #L144 - L147 were not covered by tests
}

const fn chunk_key_separator_default_zarr_v2() -> ChunkKeySeparator {
ChunkKeySeparator::Dot
}
Expand Down
20 changes: 7 additions & 13 deletions zarrs_metadata/src/v2_to_v3.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ use crate::{
},
fill_value::{FillValueFloat, FillValueFloatStringNonFinite, FillValueMetadataV3},
},
AdditionalFields, ArrayMetadataV3, GroupMetadataV3, MetadataV3,
ArrayMetadataV3, GroupMetadataV3, MetadataV3,
},
};

Expand Down Expand Up @@ -231,18 +231,12 @@ pub fn array_metadata_v2_to_v3(

let attributes = array_metadata_v2.attributes.clone();

Ok(ArrayMetadataV3::new(
shape,
data_type,
chunk_grid,
chunk_key_encoding,
fill_value,
codecs,
attributes,
vec![],
None,
AdditionalFields::default(),
))
Ok(
ArrayMetadataV3::new(shape, chunk_grid, data_type, fill_value, codecs)
.with_attributes(attributes)
.with_additional_fields(array_metadata_v2.additional_fields.clone())
.with_chunk_key_encoding(chunk_key_encoding),
)
}

/// An unsupported Zarr V2 data type error.
Expand Down
76 changes: 64 additions & 12 deletions zarrs_metadata/src/v3/array.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use chunk_key_encoding::default::DefaultChunkKeyEncodingConfiguration;
use data_type::DataTypeMetadataV3;
use derive_more::Display;
use fill_value::FillValueMetadataV3;
Expand Down Expand Up @@ -98,6 +99,7 @@ pub mod nan_representations;
/// }
/// ```
#[non_exhaustive]
#[allow(clippy::unsafe_derive_deserialize)]
#[derive(Serialize, Deserialize, Clone, PartialEq, Debug, Display)]
#[display("{}", serde_json::to_string(self).unwrap_or_default())]
pub struct ArrayMetadataV3 {
Expand Down Expand Up @@ -154,21 +156,33 @@ pub struct ArrayMetadataV3 {
}

impl ArrayMetadataV3 {
/// Create a new array metadata.
#[allow(clippy::too_many_arguments)]
/// Create new Zarr V3 array metadata.
///
/// Defaults to:
/// - `default` chunk key encoding with the '/' separator,
/// - empty attributes,
/// - no dimension names,
/// - no storage transformers, and
/// - no additional fields.
#[must_use]
pub fn new(
shape: ArrayShape,
data_type: DataTypeMetadataV3,
chunk_grid: MetadataV3,
chunk_key_encoding: MetadataV3,
data_type: DataTypeMetadataV3,
fill_value: FillValueMetadataV3,
codecs: Vec<MetadataV3>,
attributes: serde_json::Map<String, serde_json::Value>,
storage_transformers: Vec<MetadataV3>,
dimension_names: Option<Vec<DimensionName>>,
additional_fields: AdditionalFields,
) -> Self {
let chunk_key_encoding = unsafe {
// SAFETY: The default chunk key encoding configuration is valid JSON.
MetadataV3::new_with_serializable_configuration(
crate::v3::array::chunk_key_encoding::default::IDENTIFIER,
&DefaultChunkKeyEncodingConfiguration {
separator: crate::ChunkKeySeparator::Slash,
},
)
.unwrap_unchecked()
};

Self {
zarr_format: monostate::MustBe!(3u64),
node_type: monostate::MustBe!("array"),
Expand All @@ -178,10 +192,48 @@ impl ArrayMetadataV3 {
chunk_key_encoding,
fill_value,
codecs,
attributes,
storage_transformers,
dimension_names,
additional_fields,
attributes: serde_json::Map::default(),
storage_transformers: Vec::default(),
dimension_names: None,
additional_fields: AdditionalFields::default(),
}
}

/// Set the user attributes.
#[must_use]
pub fn with_attributes(
mut self,
attributes: serde_json::Map<String, serde_json::Value>,
) -> Self {
self.attributes = attributes;
self
}

/// Set the additional fields.
#[must_use]
pub fn with_additional_fields(mut self, additional_fields: AdditionalFields) -> Self {
self.additional_fields = additional_fields;
self
}

/// Set the chunk key encoding.
#[must_use]
pub fn with_chunk_key_encoding(mut self, chunk_key_encoding: MetadataV3) -> Self {
self.chunk_key_encoding = chunk_key_encoding;
self
}

/// Set the dimension names.
#[must_use]
pub fn with_dimension_names(mut self, dimension_names: Option<Vec<DimensionName>>) -> Self {
self.dimension_names = dimension_names;
self
}

/// Set the storage transformers.
#[must_use]
pub fn with_storage_transformers(mut self, storage_transformers: Vec<MetadataV3>) -> Self {
self.storage_transformers = storage_transformers;
self
}
}