From bb18b0a46a510789514a96801205503eb8ac3178 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Mon, 21 Feb 2022 15:03:54 +0100 Subject: [PATCH 01/11] Update reademe examples --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 7968414f..517da8e4 100644 --- a/README.md +++ b/README.md @@ -28,21 +28,21 @@ The oemetadata example and oemetadata template are licensed under [Creative Comm ## Usage Examples ``` -from metadata.v150.example import OEMETADATA_v150_EXAMPLE +from metadata.latest.example import OEMETADATA_v150_EXAMPLE -print(OEMETADATA_v150_EXAMPLE) +print(OEMETADATA_LATEST_EXAMPLE) ``` ``` -from metadata.v150.schema import OEMETADATA_v150_SCHEMA +from metadata.latest.schema import OEMETADATA_v150_SCHEMA -print(OEMETADATA_v150_SCHEMA) +print(OEMETADATA_LATEST_SCHEMA) ``` ``` -from metadata.v150.template import OEMETADATA_v150_TEMPLATE +from metadata.latest.template import OEMETADATA_v150_TEMPLATE -print(OEMETADATA_v150_TEMPLATE) +print(OEMETADATA_LATEST_TEMPLATE) ``` ## Contributing From 0404b2afebd46edaf4146313b93186a83ad760d1 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Mon, 21 Feb 2022 15:04:50 +0100 Subject: [PATCH 02/11] Update reademe examples --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 517da8e4..2d756d78 100644 --- a/README.md +++ b/README.md @@ -28,19 +28,19 @@ The oemetadata example and oemetadata template are licensed under [Creative Comm ## Usage Examples ``` -from metadata.latest.example import OEMETADATA_v150_EXAMPLE +from metadata.latest.example import OEMETADATA_LATEST_EXAMPLE print(OEMETADATA_LATEST_EXAMPLE) ``` ``` -from metadata.latest.schema import OEMETADATA_v150_SCHEMA +from metadata.latest.schema import OEMETADATA_LATEST_SCHEMA print(OEMETADATA_LATEST_SCHEMA) ``` ``` -from metadata.latest.template import OEMETADATA_v150_TEMPLATE +from metadata.latest.template import OEMETADATA_LATEST_TEMPLATE print(OEMETADATA_LATEST_TEMPLATE) ``` From 50a0358a70da75bc3169b165886d0a24cf81c4e5 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Mon, 21 Feb 2022 15:11:48 +0100 Subject: [PATCH 03/11] Add instructions on how to do a new realse --- RELEASE_PROCEDURE.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 RELEASE_PROCEDURE.md diff --git a/RELEASE_PROCEDURE.md b/RELEASE_PROCEDURE.md new file mode 100644 index 00000000..32e2c691 --- /dev/null +++ b/RELEASE_PROCEDURE.md @@ -0,0 +1,17 @@ +# Release prozedur + ## Pre-Release +- View the GitHub project for the next release and check the +Status of all issues and PR's assigned to the project. +If there are any open issues or PR's, decide if you want to include them +them to be included in the upcoming release. Make sure that all changes are +are included in the development branch before proceeding to the next steps. +## Release +- 1 Checkout a new branch from devolop branch named: release/oem-vX.X.X +- 2 Update the following files: + - Make sure all changes are documented in the changelog.md. + - Set the correct version number in the setup.py file. +- 3 Commit and Push the changes. +- 4 Create the Pull Request via the GitHub website and merge changes to +develop then merge develop in master branch. +- 5 Create a new release via the GitHub website and tag the master branch. +- 6 Update the PyPi (pip) Package [oep-metadata](https://pypi.org/project/oep-metadata/) \ No newline at end of file From b6ed891a4628521ef1510a40adc51f1a254950f4 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Mon, 21 Feb 2022 15:14:06 +0100 Subject: [PATCH 04/11] Fix title --- RELEASE_PROCEDURE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RELEASE_PROCEDURE.md b/RELEASE_PROCEDURE.md index 32e2c691..1f83636f 100644 --- a/RELEASE_PROCEDURE.md +++ b/RELEASE_PROCEDURE.md @@ -1,4 +1,4 @@ -# Release prozedur +# Release procedure ## Pre-Release - View the GitHub project for the next release and check the Status of all issues and PR's assigned to the project. From 9faf37a6054c7f77da4359ebe59d8db8acb69447 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Mon, 21 Feb 2022 15:18:08 +0100 Subject: [PATCH 05/11] Mention the release procedure in the readme --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 2d756d78..745a0ec9 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,10 @@ print(OEMETADATA_LATEST_TEMPLATE) For further contributing infos and conventions see: [CONTRIBUTING.md](./CONTRIBUTING.md) -## Make PyPI release: +## Release a new version +See the complete instructions in the [RELEASE_PROCEDURE](./RELEASE_PROCEDURE.md). + +### Make PyPI release: First bump version in setup.py, then: From d0bcbb5ce401755a7597c07313a09c185bfc4099 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Wed, 23 Feb 2022 11:49:10 +0100 Subject: [PATCH 06/11] add directory for version 1.5.1 and add files --- metadata/v150/schema_old.json | 711 +++++++++++++++++ metadata/v151/README.md | 16 + metadata/v151/__init__.py | 0 metadata/v151/context.json | 56 ++ metadata/v151/example.json | 399 ++++++++++ metadata/v151/example.py | 5 + metadata/v151/metadata_key_description.md | 182 +++++ metadata/v151/schema.json | 926 ++++++++++++++++++++++ metadata/v151/schema.py | 5 + metadata/v151/template.json | 196 +++++ metadata/v151/template.py | 5 + 11 files changed, 2501 insertions(+) create mode 100644 metadata/v150/schema_old.json create mode 100644 metadata/v151/README.md create mode 100644 metadata/v151/__init__.py create mode 100644 metadata/v151/context.json create mode 100644 metadata/v151/example.json create mode 100644 metadata/v151/example.py create mode 100644 metadata/v151/metadata_key_description.md create mode 100644 metadata/v151/schema.json create mode 100644 metadata/v151/schema.py create mode 100644 metadata/v151/template.json create mode 100644 metadata/v151/template.py diff --git a/metadata/v150/schema_old.json b/metadata/v150/schema_old.json new file mode 100644 index 00000000..9d56e7b9 --- /dev/null +++ b/metadata/v150/schema_old.json @@ -0,0 +1,711 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://raw.githubusercontent.com/OpenEnergyPlatform/oemetadata/master/oemetadata/v150/schema.json", + "description": "Open Energy Plaftorm (OEP) metadata schema v1.5.0", + "type": "object", + "properties": { + "@context": { + "description": "Explanation of metadata keys in ontology terms. Example: https://raw.githubusercontent.com/LOD-GEOSS/databus-snippets/master/oep_metadata/context.jsonld", + "type": [ + "string", + "null" + ] + }, + "name": { + "description": "File name or database table name. Example: oep_metadata_table_example_v15", + "type": [ + "string", + "null" + ] + }, + "title": { + "description": "Human readable title. Example: Metadata Example Table", + "type": [ + "string", + "null" + ] + }, + "id": { + "description": "Uniform Resource Identifier (URI) that unambiguously identifies the resource. This can be a URL on the data set. It can also be a Digital Object Identifier (DOI). Example: https://example.com", + "type": [ + "string", + "null" + ] + }, + "@id": { + "description": "Uniform Resource Identifier (URI) that links the resource via the databus", + "type": [ + "string", + "null" + ] + }, + "description": { + "description": "A description of the package. It should be usable as summary information for the entire package that is described by the metadata. Example: Example table used to illustrate the metadata structure and meaning", + "type": [ + "string", + "null" + ] + }, + "subject": { + "description": "Reference the topic of the resource in ontology terms", + "type": [ + "string", + "null" + ] + }, + "language": { + "description": "Language used within the described data structures (e.g. titles, descriptions). The language key can be repeated if more languages are used. Standard: IETF (BCP47). Example: [en-GB, de-DE, fr-FR]", + "type": "array", + "items": { + "type": [ + "string", + "null" + ] + } + }, + "keywords": { + "description": "An array of string keywords to assist users searching for the package in catalogs. Example: [example, template, test]", + "type": "array", + "items": { + "type": [ + "string", + "null" + ] + } + }, + "publicationDate": { + "description": "Date of publishing. Date Format is ISO 8601 (YYYY-MM-DD). Example: 2019-02-06", + "type": [ + "string", + "null" + ] + }, + "context": { + "description": "Object. Contains name-value-pairs that describe the general setting, evironment or project leading to the creation or maintenance of this dataset.", + "type": "object", + "properties": { + "homepage": { + "description": "URL of project. Example: https://openenergy-platform.org/", + "type": [ + "string", + "null" + ] + }, + "documentation": { + "description": "URL of project documentation. Example: https://github.com/OpenEnergyPlatform/metadata/wiki/Metadata-Description", + "type": [ + "string", + "null" + ] + }, + "sourceCode": { + "description": "Url of project source code. Example: https://github.com/OpenEnergyPlatform", + "type": [ + "string", + "null" + ] + }, + "contact": { + "description": "Reference to the creator or maintainer of the data set. Example: contact@example.com", + "type": [ + "string", + "null" + ] + }, + "grantNo": { + "description": "In a publicly funded Project: the identifying grant number. Example: 01AB2345", + "type": [ + "string", + "null" + ] + }, + "fundingAgency": { + "description": "In a funded Project: The name of the funding agency. Example: Bundesministerium für Wirtschaft und Energie", + "type": [ + "string", + "null" + ] + }, + "fundingAgencyLogo": { + "description": "In a publicly funded Project: A link to the Logo of the funding agency. Example: https://www.innovation-beratung-foerderung.de/INNO/Redaktion/DE/Bilder/Titelbilder/titel_foerderlogo_bmwi.jpg?__blob=poster&v=2", + "type": [ + "string", + "null" + ] + }, + "publisherLogo": { + "description": "Link to the logo of the publishing institution. Example: https://reiner-lemoine-institut.de//wp-content/uploads/2015/09/rlilogo.png", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + }, + "spatial": { + "description": "Object. Contains name-value-pairs describing the spatial context of the contained data.", + "type": "object", + "properties": { + "location": { + "description": "In the case of data where the location can be described as a point. May come as coordinates, URI or addresses with street, house number and zip code. Example: 52.433509, 13.535855", + "type": [ + "string", + "null" + ] + }, + "extent": { + "description": "Covered area. May be the name of a region, or the geometry of a bounding box. Example: Europe", + "type": [ + "string", + "null" + ] + }, + "resolution": { + "description": "Pixel size in case of a regular raster image. Reference to administrative level or other spatial division that is present as the smallest spatially distinguished unit size. Example: 30 m", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + }, + "temporal": { + "description": "Temporal object. Time period covered in the data. Temporal information should either contain a \"referenceDate\" or the keys describing a time series; in rare cases both. Use null for the ones that don't apply.", + "type": "object", + "properties": { + "referenceDate": { + "description": "Base year, month or day. Point in time for which the data is meant to be accurate. A census will generally have a reference year. A satellite image will have a reference date. Date Format is ISO 8601. Example: 2016-01-01", + "type": [ + "string", + "null" + ] + }, + "timeseries": { + "description": "Times series object in temporal object, contains start, end, resolution, alignment and aggregation type properties.", + "type": [ + "array", + "null" + ], + "properties": { + "start": { + "description": "The beginning point in time of a time series. Example: 2019-02-06T10:12:04+00:00", + "type": [ + "string", + "null" + ] + }, + "end": { + "description": "The end point in time of a time series. Example: 2019-02-07T10:12:04+00:00", + "type": [ + "string", + "null" + ] + }, + "resolution": { + "description": "The time span between individual points of information in a time series. Example: 30 s", + "type": [ + "string", + "null" + ] + }, + "alignment": { + "description": "Indicator whether stamps in a time series are left, right or middle. \"null\" if there is no time series. Example: left", + "type": [ + "string", + "null" + ] + }, + "aggregationType": { + "description": "Indicates whether the values are a sum, average or current. Example: sum", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "sources": { + "description": "List of source objects. Each object has all name-value-pairs.", + "type": "array", + "items": { + "description": "Source object in list of source objects. Each object has all name-value-pairs.", + "type": "object", + "properties": { + "title": { + "description": "Human readable title of the source, e.g. document title or organisation name. Example: IPCC Fifth Assessment Report", + "type": [ + "string", + "null" + ] + }, + "description": { + "description": "Free text description of the data set. Example: Scientific climate change report by the UN", + "type": [ + "string", + "null" + ] + }, + "path": { + "description": "URL to original source. Example: https://www.ipcc.ch/site/assets/uploads/2018/02/ipcc_wg3_ar5_full.pdf", + "type": [ + "string", + "null" + ] + }, + "licenses": { + "description": "The license(s) under which the source(s) is/are provided. List of objects.", + "type": "array", + "items": { + "description": "A license object under which the described source is provided. Each object has all name-value-pairs.", + "type": "object", + "properties": { + "name": { + "description": "SPDX identifier: Example: ODbL-1.0", + "type": [ + "string", + "null" + ] + }, + "title": { + "description": "Official (human readable) title. Example: Open Data Commons Open Database License 1.0", + "type": [ + "string", + "null" + ] + }, + "path": { + "description": "A link to the license. Example: https://opendatacommons.org/licenses/odbl/1-0/index.html", + "type": [ + "string", + "null" + ] + }, + "instruction": { + "description": "Short description of rights and restrictions. Example: You are free to share and change, but you must attribute, and share derivations under the same license.", + "type": [ + "string", + "null" + ] + }, + "attribution": { + "description": "Copyrightholder of the source. Example: © Intergovernmental Panel on Climate Change 2014", + "type": [ + "string", + "null" + ] + } + } + } + } + }, + "additionalProperties": false + } + }, + "licenses": { + "description": "The license(s) under which the described package is provided. List of objects.", + "type": "array", + "items": { + "description": "A license object under which the described package is provided. Each object has all name-value-pairs.", + "type": "object", + "properties": { + "name": { + "description": "SPDX identifier. Example: ODbL-1.0", + "type": [ + "string", + "null" + ] + }, + "title": { + "description": "Official (human readable) title. Example: Open Data Commons Open Database License 1.0", + "type": [ + "string", + "null" + ] + }, + "path": { + "description": "A url-or-path string, that is a fully qualified HTTP address, or a relative POSIX path (see the url-or-path definition in Data Resource for details). Example: https://opendatacommons.org/licenses/odbl/1-0/index.html", + "type": [ + "string", + "null" + ] + }, + "instruction": { + "description": "Short description of rights and restrictions. Example: You are free to share and change, but you must attribute, and share derivations under the same license.", + "type": [ + "string", + "null" + ] + }, + "attribution": { + "description": "Copyrightholder of the produced data set. Example: © Reiner Lemoine Institut", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + } + }, + "contributors": { + "description": "The people or organizations who contributed to this data package. List of objects.", + "type": "array", + "items": { + "description": "A person or organizations who contributed to this data package. Each object refers to one contributor. Every contributor must have a title and property. A path, email, role and organization properties are optional extras.", + "type": "object", + "properties": { + "title": { + "description": "Name/title of the contributor (name for a person, name or title for an organization). Example: Jon Doe", + "type": [ + "string", + "null" + ] + }, + "email": { + "description": "E-mail address of the contributor. Example: contact@example.com", + "type": [ + "string", + "null" + ] + }, + "date": { + "description": "Date of the contribution. If the contribution took more than a day, use the date of the final contribiution. Date Format is ISO 8601. Example: 2016-06-16", + "type": [ + "string", + "null" + ] + }, + "object": { + "description": "Target of contribution. Which part of the package was supplied/changed. Example: Metadata", + "type": [ + "string", + "null" + ] + }, + "comment": { + "description": "Free text comment on what's been done. Example: Fixed a typo in the title", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + } + }, + "resources": { + "description": "Resources, described as a list of data resource format objects.", + "type": "array", + "items": { + "description": "The data resource format describes a data resource as an individual file or table.", + "type": "object", + "properties": { + "profile": { + "description": "A string identifying the profile of this descriptor as per the profiles specification. This information is retained in order to comply with the \"Tabular Data Package\" standard. If at all in doubt the value should read \"tabular-data-resource\". Example: tabular-data-resource", + "type": [ + "string", + "null" + ] + }, + "name": { + "description": "A resource MUST contain a name unique to amongst all resources in this data package. To comply with the data package standard it must consist of only lowercase alphanumeric character plus \".\", \"-\" and \"_\". It may not start with a number. In a database this will be the name of the table within its containing schema. It would be usual for the name to correspond to the file name (minus the file-extension) of the data file the resource describes. Example: sandbox.example_table", + "type": [ + "string", + "null" + ] + }, + "path": { + "description": "A url-or-path string, that should be a permanent http(s) address or other path directly linking to the resource. Example: directly linking to the resource. https://openenergy-platform.org/dataedit/view/openstreetmap/osm_deu_roads", + "type": [ + "string", + "null" + ] + }, + "format": { + "description": "\"csv\", \"xls\", \"json\" etc. would be expected to be the standard file extension for this type of resource. When you upload your data to the OEDB, in the shown metadata string, the format will be changed accordingly to \"PostgreSQL\", since the data there are stored in a database. Example: csv", + "type": [ + "string", + "null" + ] + }, + "encoding": { + "description": "Specifies the character encoding of the resource's data file. The values should be one of the \"Preferred MIME Names\" for a character encoding registered with IANA. If no value for this key is specified then the default is UTF-8. Example: UTF-8", + "type": [ + "string", + "null" + ] + }, + "schema": { + "description": "Object containing fields, primary key and for foreign keys. Describes the structure of the present data.", + "type": "object", + "properties": { + "fields": { + "description": "List of field objects.", + "type": "array", + "items": { + "description": "Field object. Every object describes a column and provides name, description, type and unit.", + "type": "object", + "properties": { + "name": { + "description": "Name string unique within its scope. Example: year", + "type": [ + "string", + "null" + ] + }, + "description": { + "description": "Free-text describing the field. Example: Reference year for which the data were collected.", + "type": [ + "string", + "null" + ] + }, + "type": { + "description": "Data type of the field. In case of a geom-column in a database, also indicate the shape and CRS. Example: geometry(Point, 4326)", + "type": [ + "string", + "null" + ] + }, + "is_about": { + "description": "Ontology URI to describe the column header", + "type": [ + "string", + "null" + ] + }, + "value_reference": { + "description": "Ontology URI for an extended description of the values in the column", + "type": [ + "string", + "null" + ] + }, + "unit": { + "description": "Unit, preferably SI-Unit, that values in this field are mapped to. If \"unit\" doesn't apply to a field, use \"null\". Example: MW", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + } + }, + "primaryKey": { + "description": "A primary key is a field or set of fields that uniquely identifies each row in the table. It's recorded as a list of strings, since it is possible to define the primary key as made up of several columns. Example: id", + "type": "array", + "items": { + "type": [ + "string", + "null" + ] + } + }, + "foreignKeys": { + "description": "List of foreign keys.", + "type": "array", + "items": { + "description": "A foreign key is a field that refers to a column in another table.", + "type": "object", + "properties": { + "fields": { + "description": "The column (as list of items) in the table that is constrainted by the foreign key. Example: version", + "type": "array", + "items": { + "type": [ + "string", + "null" + ] + } + }, + "reference": { + "description": "The reference to the foreign table.", + "type": "object", + "properties": { + "resource": { + "description": "The foreign resource (table). Example: schema.table", + "type": [ + "string", + "null" + ] + }, + "fields": { + "description": "The foreign resource column. List of fields. Example: version", + "type": "array", + "items": { + "type": [ + "string", + "null" + ] + } + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + } + } + }, + "additionalProperties": false + }, + "dialect": { + "description": "Object. A CSV Dialect defines a simple format to describe the various dialects of CSV files in a language agnostic manner. In case of a database, the values in the containing fields are \"null\".", + "type": "object", + "properties": { + "delimiter": { + "description": "Specifies the character sequence which should separate fields (aka columns). Common characters are \",\" (comma), \".\" (point) and \"\t\" (tab). Example: ,", + "type": [ + "string", + "null" + ] + }, + "decimalSeparator": { + "description": "Symbol used to separate the integer part from the fractional part of a number written in decimal form. Depending on language and region this symbol can be \".\" or \",\". Example: .", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + } + }, + "review": { + "description": "Data uploaded through the OEP needs to go through review. The review will cover the areas described here: https://github.com/OpenEnergyPlatform/data-preprocessing/wiki and carried out by a team of the platform. The review itself is documented at the specified path and a badge is rewarded with regards to completeness.", + "type": "object", + "properties": { + "path": { + "description": "A URL or path string, that should be a permanent http(s) address directly linking to the documented review. Example: https://www.example.com", + "type": [ + "string", + "null" + ] + }, + "badge": { + "description": "A badge of either Bronze, Silver, Gold or Platin is used to label the given metadata based on its quality. Example: Platin", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + }, + "metaMetadata": { + "description": "Object. Description about the metadata themselves, their format, version and license. These fields should already be provided when you’re filling out your metadata.", + "type": "object", + "properties": { + "metadataVersion": { + "description": "Type and version number of the metadata. Example: OEP-1.5", + "type": [ + "string", + "null" + ] + }, + "metadataLicense": { + "description": "Object describing the license of the provided metadata.", + "type": "object", + "properties": { + "name": { + "description": "SPDX identifier. Example: CC0-1.0", + "type": [ + "string", + "null" + ] + }, + "title": { + "description": "Official (human readable) license title. Example: Creative Commons Zero v1.0 Universal", + "type": [ + "string", + "null" + ] + }, + "path": { + "description": "Url or path string, that is a fully qualified HTTP address. Example: https://creativecommons.org/publicdomain/zero/1.0/", + "type": [ + "string", + "null" + ] + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "_comment": { + "description": "Object. The “_comment”-section is used as a self-description of the final metadata-file. It is text, intended for humans and can include a link to the metadata documentation(s), required value formats and similar remarks. The comment section has no fix structure or mandatory values, but a useful self-description, similar to the one depicted here, is encouraged.", + "type": "object", + "properties": { + "metadata": { + "description": "Reference to the metadata documentation in use. Example: Metadata documentation and explanation (https://github.com/OpenEnergyPlatform/organisation/wiki/metadata)", + "type": [ + "string", + "null" + ] + }, + "dates": { + "description": "Comment on data/time format. Example: Dates and time must follow the ISO8601 including time zone (YYYY-MM-DD or YYYY-MM-DDThh:mm:ss±hh)", + "type": [ + "string", + "null" + ] + }, + "units": { + "description": "Comment on units. Example: If you must use units in cells (which is discouraged), leave a space between numbers and units (100 m)", + "type": [ + "string", + "null" + ] + }, + "languages": { + "description": "Comment on language format. Example: Languages must follow the IETF (BCP47) format (en-GB, en-US, de-DE)", + "type": [ + "string", + "null" + ] + }, + "licenses": { + "description": "Reference to license format. Example: License name must follow the SPDX License List (https://spdx.org/licenses/)", + "type": [ + "string", + "null" + ] + }, + "review": { + "description": "Reference to review documentation. Example: Following the OEP Data Review (https://github.com/OpenEnergyPlatform/data-preprocessing/wiki)", + "type": [ + "string", + "null" + ] + }, + "null": { + "description": "Feel free to add more descriptive comments. Like \"null\". Example: If a field is not applicable just enter \"null\"", + "type": [ + "string", + "null" + ] + }, + "todo": { + "description": "If an applicable value is not yet available and will be inserted later on use: 'todo' ", + "type": [ + "string", + "null" + ] + } + } + } + }, + "additionalProperties": false +} \ No newline at end of file diff --git a/metadata/v151/README.md b/metadata/v151/README.md new file mode 100644 index 00000000..dee9fa23 --- /dev/null +++ b/metadata/v151/README.md @@ -0,0 +1,16 @@ +OpenEnergyMetadata +OpenEnergyPlatform + + +# Open Energy Metadata (OEM) - Latest-Release + +This version of the OEM represents the latest released version. This release is fully integrated +into the OEP. + +The OEM contains the following files: + +* [template.json](https://github.com/OpenEnergyPlatform/oemetadata/blob/master/metadata/latest/template.json) contains an empty metadata string with all fields. +* [metadata_key_description.md](https://github.com/OpenEnergyPlatform/oemetadata/blob/master/metadata/latest/metadata_key_description.md) contains a full description of each metadata key and an example. +* [example.json](https://github.com/OpenEnergyPlatform/oemetadata/blob/master/metadata/latest/example.json) contains a basic metadata example. + +For further information see the [Changelog](https://github.com/OpenEnergyPlatform/oemetadata/blob/master/CHANGELOG.md). diff --git a/metadata/v151/__init__.py b/metadata/v151/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/metadata/v151/context.json b/metadata/v151/context.json new file mode 100644 index 00000000..2875c27f --- /dev/null +++ b/metadata/v151/context.json @@ -0,0 +1,56 @@ +{ + "@context": { + "path": "@id", + "rdfs": "http://www.w3.org/2000/01/rdf-schema#", + "dct": "http://purl.org/dc/terms/", + "dc": "http://purl.org/dc/elements/1.1/", + "xsd": "http://www.w3.org/2001/XMLSchema#", + "oeo": "http://openenergy-platform.org/ontology/oeo/", + "csvw": "http://www.w3.org/ns/csvw#", + "obo": "http://purl.obolibrary.org/obo/", + "title": { + "@id": "dct:title", + "@type": "xsd:string" + }, + "description": { + "@id": "dct:description", + "@type": "xsd:string" + }, + "comment": { + "@id": "rdfs:comment", + "@type": "xsd:string" + }, + "fields": { + "@id": "csvw:column", + "@type": "@id" + }, + "resources": { + "@id": "csvw:table", + "@type": "@id" + }, + "schema": { + "@id": "csvw:tableSchema", + "@type": "@id" + }, + "subject": { + "@id": "dc:subject", + "@type": "@id" + }, + "name": { + "@id": "rdfs:label", + "@type": "xsd:string" + }, + "type": { + "@id": "csvw:datatype", + "@type": "xsd:string" + }, + "unit": { + "@id": "oeo:OEO_00040010", + "@type": "xsd:string" + }, + "about": { + "@id": "obo:IAO_0000136", + "@type": "@id" + } + } +} \ No newline at end of file diff --git a/metadata/v151/example.json b/metadata/v151/example.json new file mode 100644 index 00000000..1bb3ef20 --- /dev/null +++ b/metadata/v151/example.json @@ -0,0 +1,399 @@ +{ + "name": "oep_metadata_table_example_v151", + "title": "Example title for metadata example - Version 1.5.1", + "id": "http://openenergyplatform.org/dataedit/view/model_draft/oep_metadata_table_example_v151", + "description": "This is an metadata example for example data. There is a corresponding table on the OEP for each metadata version.", + "language": [ + "en-GB", + "en-US", + "de-DE", + "fr-FR" + ], + "subject": [ + { + "name": "energy", + "path": "https://openenergy-platform.org/ontology/oeo/OEO_00000150" + }, + { + "name": "test dataset", + "path": "https://openenergy-platform.org/ontology/oeo/OEO_00000408" + } + ], + "keywords": [ + "energy", + "example", + "template", + "test" + ], + "publicationDate": "2022-02-15", + "context": { + "homepage": "https://reiner-lemoine-institut.de/lod-geoss/", + "documentation": "https://openenergy-platform.org/tutorials/jupyter/OEMetadata/", + "sourceCode": "https://github.com/OpenEnergyPlatform/oemetadata/tree/master", + "contact": "https://github.com/Ludee", + "grantNo": "03EI1005", + "fundingAgency": "Bundesministerium für Wirtschaft und Klimaschutz", + "fundingAgencyLogo": "https://commons.wikimedia.org/wiki/File:BMWi_Logo_2021.svg#/media/File:BMWi_Logo_2021.svg", + "publisherLogo": "https://reiner-lemoine-institut.de//wp-content/uploads/2015/09/rlilogo.png" + }, + "spatial": { + "location": null, + "extent": "europe", + "resolution": "100 m" + }, + "temporal": { + "referenceDate": "2016-01-01", + "timeseries": [ + { + "start": "2017-01-01T00:00+01", + "end": "2017-12-31T23:00+01", + "resolution": "1 h", + "alignment": "left", + "aggregationType": "sum" + }, + { + "start": "2018-01-01T00:00+01", + "end": "2019-06-01T23:00+01", + "resolution": "15 min", + "alignment": "right", + "aggregationType": "sum" + } + ] + }, + "sources": [ + { + "title": "OpenEnergyPlatform Metadata Example", + "description": "Metadata description", + "path": "https://github.com/OpenEnergyPlatform", + "licenses": [ + { + "name": "CC0-1.0", + "title": "Creative Commons Zero v1.0 Universal", + "path": "https://creativecommons.org/publicdomain/zero/1.0/legalcode", + "instruction": "You are free: To Share, To Create, To Adapt", + "attribution": "© Reiner Lemoine Institut" + } + ] + }, + { + "title": "OpenStreetMap", + "description": "A collaborative project to create a free editable map of the world", + "path": "https://www.openstreetmap.org/", + "licenses": [ + { + "name": "ODbL-1.0", + "title": "Open Data Commons Open Database License 1.0", + "path": "https://opendatacommons.org/licenses/odbl/1.0/index.html", + "instruction": "You are free: To Share, To Create, To Adapt; As long as you: Attribute, Share-Alike, Keep open!", + "attribution": "© OpenStreetMap contributors" + } + ] + } + ], + "licenses": [ + { + "name": "ODbL-1.0", + "title": "Open Data Commons Open Database License 1.0", + "path": "https://opendatacommons.org/licenses/odbl/1.0/", + "instruction": "You are free: To Share, To Create, To Adapt; As long as you: Attribute, Share-Alike, Keep open!", + "attribution": "© Reiner Lemoine Institut © OpenStreetMap contributors" + } + ], + "contributors": [ + { + "title": "Ludee", + "email": null, + "date": "2016-06-16", + "object": "metadata", + "comment": "Create metadata" + }, + { + "title": "Ludee", + "email": null, + "date": "2016-11-22", + "object": "metadata", + "comment": "Update metadata" + }, + { + "title": "Ludee", + "email": null, + "date": "2016-11-22", + "object": "metadata", + "comment": "Update header and license" + }, + { + "title": "Ludee", + "email": null, + "date": "2017-03-16", + "object": "metadata", + "comment": "Add license to source" + }, + { + "title": "Ludee", + "email": null, + "date": "2017-03-28", + "object": "metadata", + "comment": "Add copyright to source and license" + }, + { + "title": "Ludee", + "email": null, + "date": "2017-05-30", + "object": "metadata", + "comment": "Release metadata version 1.3" + }, + { + "title": "Ludee", + "email": null, + "date": "2017-06-26", + "object": "metadata", + "comment": "Move referenceDate into temporal and remove array" + }, + { + "title": "Ludee", + "email": null, + "date": "2018-07-19", + "object": "metadata", + "comment": "Start metadata version 1.4" + }, + { + "title": "Ludee", + "email": null, + "date": "2018-07-26", + "object": "data", + "comment": "Rename table and files" + }, + { + "title": "Ludee", + "email": null, + "date": "2018-10-18", + "object": "metadata", + "comment": "Add contribution object" + }, + { + "title": "christian-rli", + "email": null, + "date": "2018-10-18", + "object": "metadata", + "comment": "Add datapackage compatibility" + }, + { + "title": "Ludee", + "email": null, + "date": "2018-11-02", + "object": "metadata", + "comment": "Release metadata version 1.4" + }, + { + "title": "christian-rli", + "email": null, + "date": "2019-02-05", + "object": "metadata", + "comment": "Apply template structure to example" + }, + { + "title": "Ludee", + "email": null, + "date": "2019-03-22", + "object": "metadata", + "comment": "Hotfix foreignKeys" + }, + { + "title": "Ludee", + "email": null, + "date": "2019-07-09", + "object": "metadata", + "comment": "Release metadata version OEP-1.3.0" + }, + { + "title": "Ludee", + "email": null, + "date": "2021-11-15", + "object": "metadata", + "comment": "Release metadata version OEP-1.5.0" + }, + { + "title": "Ludee", + "email": null, + "date": "2022-02-15", + "object": "metadata", + "comment": "Release metadata version OEP-1.5.1" + } + ], + "resources": [ + { + "profile": "tabular-data-resource", + "name": "model_draft.oep_metadata_table_example_v151", + "path": "http://openenergyplatform.org/dataedit/view/model_draft/oep_metadata_table_example_v151", + "format": "PostgreSQL", + "encoding": "UTF-8", + "schema": { + "fields": [ + { + "name": "id", + "description": "Unique identifier", + "type": "serial", + "unit": null, + "isAbout": [ + { + "name": null, + "path": null + } + ], + "valueReference": [ + { + "value": null, + "name": null, + "path": null + } + ] + }, + { + "name": "name", + "description": "Example name", + "type": "text", + "unit": null, + "isAbout": [ + { + "name": "written name", + "path": "https://openenergy-platform.org/ontology/oeo/IAO_0000590" + } + ], + "valueReference": [ + { + "value": null, + "name": null, + "path": null + } + ] + }, + { + "name": "type", + "description": "Type of wind farm", + "type": "text", + "unit": null, + "isAbout": [ + { + "name": "wind farm", + "path": "https://openenergy-platform.org/ontology/oeo/OEO_00000447" + } + ], + "valueReference": [ + { + "value": "onshore ", + "name": "onshore wind farm", + "path": "https://openenergy-platform.org/ontology/oeo/OEO_00000311" + }, + { + "value": "offshore ", + "name": "offshore wind farm", + "path": "https://openenergy-platform.org/ontology/oeo/OEO_00000308" + } + ] + }, + { + "name": "year", + "description": "Reference year", + "type": "integer", + "unit": null, + "isAbout": [ + { + "name": "year", + "path": "https://openenergy-platform.org/ontology/oeo/UO_0000036" + } + ], + "valueReference": [ + { + "value": null, + "name": null, + "path": null + } + ] + }, + { + "name": "value", + "description": "Example value", + "type": "double precision", + "unit": "MW", + "isAbout": [ + { + "name": "quantity value", + "path": "https://openenergy-platform.org/ontology/oeo/OEO_00000350" + } + ], + "valueReference": [ + { + "value": null, + "name": null, + "path": null + } + ] + }, + { + "name": "geom", + "description": "Geometry", + "type": "geometry(Point, 4326)", + "unit": null, + "isAbout": [ + { + "name": "spatial region", + "path": "https://openenergy-platform.org/ontology/oeo/BFO_0000006" + } + ], + "valueReference": [ + { + "value": null, + "name": null, + "path": null + } + ] + } + ], + "primaryKey": [ + "id" + ], + "foreignKeys": [ + { + "fields": [ + "year" + ], + "reference": { + "resource": "schema.table", + "fields": [ + "year" + ] + } + } + ] + }, + "dialect": { + "delimiter": null, + "decimalSeparator": "." + } + } + ], + "@id": "https://databus.dbpedia.org/kurzum/mastr/bnetza-mastr/01.04.00", + "@context": "https://github.com/OpenEnergyPlatform/oemetadata/blob/master/metadata/latest/context.json", + "review": { + "path": "https://github.com/OpenEnergyPlatform/data-preprocessing/issues", + "badge": "Platinum" + }, + "metaMetadata": { + "metadataVersion": "OEP-1.5.1", + "metadataLicense": { + "name": "CC0-1.0", + "title": "Creative Commons Zero v1.0 Universal", + "path": "https://creativecommons.org/publicdomain/zero/1.0/" + } + }, + "_comment": { + "metadata": "Metadata documentation and explanation (https://github.com/OpenEnergyPlatform/oemetadata)", + "dates": "Dates and time must follow the ISO8601 including time zone (YYYY-MM-DD or YYYY-MM-DDThh:mm:ss±hh)", + "units": "Use a space between numbers and units (100 m)", + "languages": "Languages must follow the IETF (BCP47) format (en-GB, en-US, de-DE)", + "licenses": "License name must follow the SPDX License List (https://spdx.org/licenses/)", + "review": "Following the OEP Data Review (https://github.com/OpenEnergyPlatform/data-preprocessing/blob/master/data-review/manual/review_manual.md)", + "null": "If not applicable use: null", + "todo": "If a value is not yet available, use: todo" + } +} \ No newline at end of file diff --git a/metadata/v151/example.py b/metadata/v151/example.py new file mode 100644 index 00000000..1d30161e --- /dev/null +++ b/metadata/v151/example.py @@ -0,0 +1,5 @@ +import json +import os + +with open(os.path.join(os.path.dirname(__file__), "example.json"), "rb") as f: + OEMETADATA_LATEST_EXAMPLE = json.loads(f.read()) diff --git a/metadata/v151/metadata_key_description.md b/metadata/v151/metadata_key_description.md new file mode 100644 index 00000000..6492a3c6 --- /dev/null +++ b/metadata/v151/metadata_key_description.md @@ -0,0 +1,182 @@ +# OEMetadata - Key Description + +This pages describes the key of **OEMetadata version 1.5.1.**
+You can have a look at an empty [template](https://github.com/OpenEnergyPlatform/metadata/blob/master/metadata/v151/template.json) and a filled out [example](https://github.com/OpenEnergyPlatform/metadata/blob/master/metadata/v151/example.json) of the metadata string. + + +## JSON Format +The JSON format offers different formats: +* key-value pair: + ```JSON + {"key":"value"} + ``` +* array: + ```JSON + {"key": + ["value","value"]} + ``` +* object {nested key}: + ```JSON + {"key": + {"key":"value"}, + {"key":"value"}} + ``` +* array of objects {nested array}: + ```JSON + {"key": [ + {"key":"value"}, + {"key":"value"}]} + ``` + + +## Metadata keys with a description and example + + +### General Keys +|#|Key |Description |Example | +|---|---|---|---| +| 1 | name | A file name or database table name. | oep_metadata_table_example_v15 | +| 2 | title | A human readable full title including author. | RLI - OEMetadata - Metadata example table | +| 3 | id | An Uniform Resource Identifier (URI) that unambiguously identifies the resource. This can be a URL on the data set. It can also be a Digital Object Identifier (DOI). | https://example.com | +| 4 | description | A description or abstract of the package. It should be usable as summary information for the entire package that is described by the metadata. | Example table used to illustrate the metadata structure and meaning. | +| 5 | language | An array of languages used within the described data structures (e.g. titles, descriptions). The language key can be repeated if more languages are used. Standard: IETF (BCP47) | en-GB, de-DE, fr-FR | +| 6 | **subject** | An array of objects with topics of the data in OEO terms. | +| 6.1 | name | The class label of the OEO terms. | energy | +| 6.2 | path | The URI of the class. | https://openenergy-platform.org/ontology/oeo/OEO_00000150 | +| 7 | keywords | An array of keywords to assist users searching for the package in catalogs. | example, template, test | +| 8 | publicationDate | A date of publishing of the data or metadata. Date format is ISO 8601 (YYYY-MM-DD). | 2019-02-06 | + +### Context Keys +|#|Key |Description |Example | +|---|---|---|---| +| 9 | **context** | An object that describes the general setting, environment, or project leading to the creation or maintenance of this dataset. In science this is can be the research project. | | +| 9.1 | homepage | A URL of the project. | https://openenergy-platform.org/ | +| 9.2 | documentation | A URL of the project documentation. | https://openenergy-platform.org/about/ | +| 9.3 | sourceCode | A URL of the projects source code. | https://github.com/OpenEnergyPlatform | +| 9.4 | contact | A reference to the creator or maintainer of the data set. It can be an email address or a GitHub handle. | contact@example.com | +| 9.5 | grantNo | An identifying grant number. In case of a publicly funded project, this number is assigned by the funding agency. | 01AB2345 | +| 9.6 | fundingAgency | A name of the entity providing the funding. This can be a government agency or a company. | Bundesministerium für Wirtschaft und Klimaschutz | +| 9.7 | fundingAgencyLogo | A URL to the logo or image of the funding agency. | https://commons.wikimedia.org/wiki/File:BMWi_Logo_2021.svg#/media/File:BMWi_Logo_2021.svg | +| 9.8 | publisherLogo | A URL to the logo of the publishing agency of data. | https://reiner-lemoine-institut.de//wp-content/uploads/2015/09/rlilogo.png | + +### Spatial and Temporal Keys +|#|Key |Description |Example | +|---|---|---|---| +| 10 | **spatial** | An object that describes the spatial context of the data it contains. | | +| 10.1 | location | A location of the data. In case of data where the location can be described as a point. May be specified as coordinates, URI or addresses with street, house number and zip code. | 52.433509, 13.535855 | +| 10.2 | extent | A covered area. May be the name of a region, or the geometry of a bounding box. | Europe | +| 10.3 | resolution | Pixel size in case of a regular raster image. Reference to administrative level or other spatial division that is present as the smallest spatially distinguished unit size. | 1 ha | +| 11 | **temporal** | An object with the time period covered in the data. Temporal information should either contain a "referenceDate" or the keys describing a time series; in rare cases both. | | +| 11.1 | referenceDate | The base year, month or day. Point in time for which the data is meant to be accurate. The census data or a satellite image will have a reference date. Date Format is ISO 8601. | 2016-01-01 | +| 11.2 | **timeseries** | An array that describes the timeseries. || +| 11.2.1 | start | The beginning point in time of a time series. | 2019-02-06T10:12:04+00:00 | +| 11.2.2 | end | The end point in time of a time series. | 2019-02-07T10:12:04+00:00 | +| 11.2.3 | resolution | The time span between individual points of information in a time series. | 30 s | +| 11.2.4 | alignment | An indicator whether stamps in a time series are left, right or middle. | left | +| 11.2.5 | aggregationType | Indicates whether the values are a sum, average or current. | sum | + +### Source Keys +|#|Key |Description |Example | +|---|---|---|---| +| 12 | **sources** | An array of objects with the used and underlying sources of the data and metadata. | | +| 12.1 | title | A human readable title of the source, a document title or organisation name. | IPCC Fifth Assessment Report | +| 12.2 | description | A free text description of the data set. | Scientific climate change report by the UN | +| 12.3 | path | A URL to the original source. | https://www.ipcc.ch/site/assets/uploads/2018/02/ipcc_wg3_ar5_full.pdf | +| 12.4 | **licenses** | An array of objects under which the source is provided. | | +| 12.4.1 | name | The [SPDX](https://spdx.org/licenses/) identifier. | ODbL-1.0 | +| 12.4.2 | title | The official (human readable) title of the license. | Open Data Commons Open Database License 1.0 | +| 12.4.3 | path | A link to the license text. | https://opendatacommons.org/licenses/odbl/1-0/index.html | +| 12.4.4 | instruction | A short description of rights and restrictions. The use of [tl;drLegal](https://tldrlegal.com/) is recommended. | You are free to share and change, but you must attribute, and share derivations under the same license. See https://tldrlegal.com/license/odc-open-database-license-(odbl) for further information. | +| 12.4.5 | attribution | The copyright owner of the **source**. If attribution licenses are used, that name must be acknowledged. | © Intergovernmental Panel on Climate Change 2014 | + +### License Keys +|#|Key |Description |Example | +|---|---|---|---| +| 13 | **licenses** | An array of objects of the license(s) under which the described package is provided. It can depend on the licenses of the sources (copyleft or share-alike) or can be granted by the creator of the data. | | +| 13.1 | name | The [SPDX](https://spdx.org/licenses/) identifier. | ODbL-1.0 | +| 13.2 | title | The official (human readable) title of the license. | Open Data Commons Open Database License 1.0 | +| 13.3 | path | A link to the license text. | https://opendatacommons.org/licenses/odbl/1-0/index.html | +| 13.4 | instruction | A short description of rights and restrictions. The use of [tl;drLegal](https://tldrlegal.com/) is recommended. | You are free to share and change, but you must attribute, and share derivations under the same license. See https://tldrlegal.com/license/odc-open-database-license-(odbl) for further information. | +| 13.5 | attribution | The copyright owner of the **data**. If attribution licenses are used, that name must be acknowledged. | © Reiner Lemoine Institut | + +### Provenience Keys +|#|Key |Description |Example | +|---|---|---|---| +| 14 | **contributors** | An array of objects of the people or organizations who contributed to the data or metadata. Each object refers to one contributor. Every contributor must have a title and property. The path, email, role and organization properties are optional. | | +| 14.1 | title | A name of the contributor. | Ludwig Hülk | +| 14.2 | email | A email address of the contributor or GitHub handle. | @Ludee | +| 14.3 | date | The date of the contribution. If the contribution took more than a day, use the date of the final contribiution. Date Format is ISO 8601. | 2016-06-16 | +| 14.4 | object | The target of the contribution. Which part of the package was supplied or changed. Can be the data, metadata or both (data and metadata). | data and metadata | +| 14.5 | comment | A free text commentary on what has been done. | Fixed a typo in the title. | + +### Resource Keys +|#|Key |Description |Example | +|---|---|---|---| +| 15 | **resources** | An array of objects of the data. It describes the data resource as an individual file or (database) table. | | +| 15.1 | profile | The profile of this descriptor according to the profiles specification. This information is retained in order to comply with the "Tabular Data Package" standard. Use "tabular-data-resource" for all tables. | tabular-data-resource | +| 15.2 | name | A name for the entire data package. To comply with the data package standard it must consist of only lowercase alphanumeric character plus ".", "-" and "_". It may not start with a number. In a database, this will be the name of the table within the schema containing it. The name usually corresponds to the file name (minus the file-extension) of the data file describing the resource. | openstreetmap.osm_deu_line | +| 15.3 | path | A URL that should be a permanent http(s) address or other path directly linking to the resource. | https://openenergy-platform.org/dataedit/view/openstreetmap/osm_deu_line | +| 15.4 | format | The file extension. 'csv', 'xls', 'json' etc. would be expected to be the standard file extension for this type of resource. When you upload your data to the OEDB, in the shown metadata string, the format will be changed accordingly to 'PostgreSQL', since the data there are stored in a database. | PostgreSQL | +| 15.5 | encoding | Specifies the character encoding of the resource's data file. The values should be one of the ["Preferred MIME Names"](https://www.iana.org/assignments/character-sets/character-sets.xhtml) for a character encoding registered with IANA. If no value for this key is specified then the default is UTF-8. | UTF-8 | + +#### Resource Keys - Schema +|#|Key |Description |Example | +|---|---|---|---| +| 15.6 | **schema** | An object that describes the structure of the present data. It contains all fields (columns of the table), the primary key and optional foreign keys. | | +| 15.6.1 | **fields** | An array of objects describing a column and providing name, description, type and unit. | | +| 15.6.1.1 | name | The name of the field. | year | +| 15.6.1.2 | description | A text describing the field. | Reference year for which the data were collected. | +| 15.6.1.3 | type | The data type of the field. In case of a geom column in a database, also indicate the shape and CRS. | geometry(Point, 4326) | +| 15.6.1.4 | unit | The unit, preferably SI-unit, that values in this field are mapped to. If 'unit' doesn't apply to a field, use 'null'. If the unit is given in a seperate field, reference this field. | MW | +| 15.6.1.5 | **isAbout** | An array of objects with describe the field in [OEO](https://openenergy-platform.org/ontology/oeo/) terms. | +| 15.6.1.5.1 | name | The class label of the OEO terms. | wind energy converting unit | +| 15.6.1.5.2 | path | The URI of the class. | https://openenergy-platform.org/ontology/oeo/OEO_00000044 | +| 15.6.1.6 | **valueReference** | An array of objects for an extended description of the values in the column in [OEO](https://openenergy-platform.org/ontology/oeo/) terms. | | +| 15.6.1.6.1 | value | The name of the value in the column. | onshore | +| 15.6.1.6.2 | name | The class label of the OEO terms. | onshore wind farm | +| 15.6.1.6.3 | path | The URI of the class. | https://openenergy-platform.org/ontology/oeo/OEO_00000311 | + +#### Resource Keys - Properties +|#|Key |Description |Example | +|---|---|---|---| +| 15.6.2 | primaryKey | A primary key is a field or set of fields that uniquely identifies each row in the table. It is recorded as an array, since it is possible to define the primary key as made up of several columns. | id | +| 15.6.3 | **foreignKeys** | A foreign key is a field that refers to a column in another table. | | +| 15.6.3.1 | fields | The column in the table that is constrainted by the foreign key. | version | +| 15.6.3.2 | **reference** | The reference to the foreign table. | | +| 15.6.3.2.1 | resource | The foreign resource (table). | schema.table | +| 15.6.3.2.2 | fields | The foreign resource column. | version | +| 15.7 | **dialect** | Object. A CSV Dialect defines a simple format to describe the various dialects of CSV files in a language agnostic manner. In case of a database, the values in the containing fields are 'null'. | | +| 15.7.1 | delimiter | The delimiter specifies the character sequence which should separate fields (columns). Common characters are "," (comma), "." (point) and "\t" (tab). | , | +| 15.7.2 | decimalSeparator | A symbol used to separate the integer part from the fractional part of a number written in decimal form. Depending on language and region this symbol can be "." or ",". | . | + +### Review Keys +|#|Key |Description |Example | +|---|---|---|---| +| 16 | @id | A Uniform Resource Identifier (URI) that links the resource via the [DBedia Databus](https://www.dbpedia.org/resources/databus/). | https://databus.dbpedia.org/kurzum/mastr/bnetza-mastr/01.04.00 | +| 17 | @context | Explanation of metadata keys in ontology terms. | https://github.com/OpenEnergyPlatform/oemetadata/blob/master/metadata/latest/context.json | +| 18. | **review** | Data uploaded through the OEP will go through a review process. The review will cover the data and metadata. It is done by the OEP community. See the [OEP Data Review](https://github.com/OpenEnergyPlatform/data-preprocessing/blob/master/data-review/manual/review_manual.md) for detailed information. The review itself is documented at the specified path and a badge is rewarded with regards to completeness. | | +| 18.1 | path | A URL that should be a permanent http(s) address directly linking to the documented review. | https://www.example.com | +| 18.2 | badge | A badge of either Bronze, Silver, Gold or Platinum is used to label the given data and metadata based on its quality. | Platinum | + +### MetaMetadata Keys +|#|Key |Description |Example | +|---|---|---|---| +| 19 | **metaMetadata** | An object that describes the metadata themselves, their format, version and license. These fields should already be provided when you are filling out your metadata. | | +| 19.1 | metadataVersion | The type and version number of the metadata. | OEP-1.5.1 | +| 19.2 | **metadataLicense** | The license of the provided metadata. | | +| 19.2.1 | name | The [SPDX](https://spdx.org/licenses/) identifier. | CC0-1.0 | +| 19.2.2 | title | The official (human readable) title of the license. | Creative Commons Zero v1.0 Universal | +| 19.2.3 | path | A link to the license text. | https://creativecommons.org/publicdomain/zero/1.0/ | + +### Comments +|#|Key |Description |Example | +|---|---|---|---| +| 20 | **_comment** | An array of objects. This section is used as a self-description of the final metadata file. It is text, intended for humans and includes a link to the metadata documentation, required value formats and similar remarks. | | +| 20.1 | metadata | Reference to the metadata documentation in use. | Metadata documentation and explanation (https://github.com/OpenEnergyPlatform/oemetadata) | +| 20.2 | dates | Comment on data and time format. | Dates and time must follow the ISO8601 including time zone (YYYY-MM-DD or YYYY-MM-DDThh:mm:ss±hh) | +| 20.3 | units | Comment on units. | Use a space between numbers and units (100 m) | +| 20.4 | languages | Comment on language format. | Languages must follow the IETF (BCP47) format (en-GB, en-US, de-DE) | +| 20.5 | licenses | Comment on license format. | License name must follow the SPDX License List (https://spdx.org/licenses/) | +| 20.6 | review | Reference to review documentation. | Following the OEP Data Review (https://github.com/OpenEnergyPlatform/data-preprocessing/blob/master/data-review/manual/review_manual.md) | +| 20.7 | null | Comment on fields that don't apply. | If not applicable use: null | +| 20.8 | todo | Comment on fields that are not yet available and will be inserted later on. | If a value is not yet available, use: todo | diff --git a/metadata/v151/schema.json b/metadata/v151/schema.json new file mode 100644 index 00000000..49ebcaaf --- /dev/null +++ b/metadata/v151/schema.json @@ -0,0 +1,926 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://raw.githubusercontent.com/OpenEnergyPlatform/oemetadata/master/oemetadata/latest/schema.json", + "description": "Open Energy Plaftorm (OEP) metadata schema latest", + "type": "object", + "properties": { + "@context": { + "description": "Explanation of metadata keys in ontology terms. Example: https://raw.githubusercontent.com/LOD-GEOSS/databus-snippets/master/oep_metadata/context.jsonld", + "type": [ + "string", + "null" + ], + "title": "@context" + }, + "name": { + "description": "File name or database table name. Example: oep_metadata_table_example_v15", + "type": [ + "string", + "null" + ], + "title": "Name" + }, + "title": { + "description": "Human readable title. Example: Metadata Example Table", + "type": [ + "string", + "null" + ], + "title": "Title" + }, + "id": { + "description": "Uniform Resource Identifier (URI) that unambiguously identifies the resource. This can be a URL on the data set. It can also be a Digital Object Identifier (DOI). Example: https://example.com", + "type": [ + "string", + "null" + ], + "title": "Id", + "readonly": true + }, + "@id": { + "description": "Uniform Resource Identifier (URI) that links the resource via the databus", + "type": [ + "string", + "null" + ], + "title": "@Id", + "readonly": true + }, + "description": { + "description": "A description of the package. It should be usable as summary information for the entire package that is described by the metadata. Example: Example table used to illustrate the metadata structure and meaning", + "type": [ + "string", + "null" + ], + "title": "Description" + }, + "subject": { + "description": "Reference the topic of the resource in ontology terms", + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "description": "Name of the OEO Class", + "type": [ + "string", + "null" + ], + "title": "Name" + }, + "path": { + "description": "Path to the OEO (URL)", + "type": [ + "string", + "null" + ], + "title": "Path", + "format": "uri" + } + }, + "additionalProperties": false, + "title": "Subject" + }, + "title": "Subject" + }, + "language": { + "description": "Language used within the described data structures (e.g. titles, descriptions). The language key can be repeated if more languages are used. Standard: IETF (BCP47). Example: [en-GB, de-DE, fr-FR]", + "type": "array", + "items": { + "type": [ + "string", + "null" + ], + "title": "Language" + }, + "title": "Language" + }, + "keywords": { + "description": "An array of string keywords to assist users searching for the package in catalogs. Example: [example, template, test]", + "type": "array", + "items": { + "type": [ + "string", + "null" + ], + "title": "Keyword" + }, + "title": "Keyword" + }, + "publicationDate": { + "description": "Date of publishing. Date Format is ISO 8601 (YYYY-MM-DD). Example: 2019-02-06", + "type": [ + "string", + "null" + ], + "title": "Publication date", + "format": "date" + }, + "context": { + "description": "Object. Contains name-value-pairs that describe the general setting, evironment or project leading to the creation or maintenance of this dataset.", + "type": "object", + "properties": { + "homepage": { + "description": "URL of project. Example: https://openenergy-platform.org/", + "type": [ + "string", + "null" + ], + "title": "Homepage", + "format": "uri" + }, + "documentation": { + "description": "URL of project documentation. Example: https://github.com/OpenEnergyPlatform/metadata/wiki/Metadata-Description", + "type": [ + "string", + "null" + ], + "title": "Documentation" + }, + "sourceCode": { + "description": "Url of project source code. Example: https://github.com/OpenEnergyPlatform", + "type": [ + "string", + "null" + ], + "title": "Source code" + }, + "contact": { + "description": "Reference to the creator or maintainer of the data set. Example: contact@example.com", + "type": [ + "string", + "null" + ], + "title": "E-Mail contact", + "format": "email" + }, + "grantNo": { + "description": "In a publicly funded Project: the identifying grant number. Example: 01AB2345", + "type": [ + "string", + "null" + ], + "title": "Grant no" + }, + "fundingAgency": { + "description": "In a funded Project: The name of the funding agency. Example: Bundesministerium für Wirtschaft und Energie", + "type": [ + "string", + "null" + ], + "title": "Funding agency" + }, + "fundingAgencyLogo": { + "description": "In a publicly funded Project: A link to the Logo of the funding agency. Example: https://www.innovation-beratung-foerderung.de/INNO/Redaktion/DE/Bilder/Titelbilder/titel_foerderlogo_bmwi.jpg?__blob=poster&v=2", + "type": [ + "string", + "null" + ], + "title": "Funding agency logo", + "format": "uri" + }, + "publisherLogo": { + "description": "Link to the logo of the publishing institution. Example: https://reiner-lemoine-institut.de//wp-content/uploads/2015/09/rlilogo.png", + "type": [ + "string", + "null" + ], + "title": "Publisher logo", + "format": "uri" + } + }, + "additionalProperties": false, + "title": "Context" + }, + "spatial": { + "description": "Object. Contains name-value-pairs describing the spatial context of the contained data.", + "type": "object", + "properties": { + "location": { + "description": "In the case of data where the location can be described as a point. May come as coordinates, URI or addresses with street, house number and zip code. Example: 52.433509, 13.535855", + "type": [ + "string", + "null" + ], + "title": "Location" + }, + "extent": { + "description": "Covered area. May be the name of a region, or the geometry of a bounding box. Example: Europe", + "type": [ + "string", + "null" + ], + "title": "Extent" + }, + "resolution": { + "description": "Pixel size in case of a regular raster image. Reference to administrative level or other spatial division that is present as the smallest spatially distinguished unit size. Example: 30 m", + "type": [ + "string", + "null" + ], + "title": "Resolution" + } + }, + "additionalProperties": false, + "title": "Spatial" + }, + "temporal": { + "description": "Temporal object. Time period covered in the data. Temporal information should either contain a \"referenceDate\" or the keys describing a time series; in rare cases both. Use null for the ones that don't apply.", + "type": "object", + "properties": { + "referenceDate": { + "description": "Base year, month or day. Point in time for which the data is meant to be accurate. A census will generally have a reference year. A satellite image will have a reference date. Date Format is ISO 8601. Example: 2016-01-01", + "type": [ + "string", + "null" + ], + "title": "Reference date", + "format": "date" + }, + "timeseries": { + "description": "Times series object in temporal object, contains start, end, resolution, alignment and aggregation type properties.", + "type": "array", + "items": { + "type": "object", + "properties": { + "start": { + "description": "The beginning point in time of a time series. Example: 2019-02-06T10:12:04+00:00", + "type": [ + "string", + "null" + ], + "title": "Start", + "format": "date-time" + }, + "end": { + "description": "The end point in time of a time series. Example: 2019-02-07T10:12:04+00:00", + "type": [ + "string", + "null" + ], + "title": "End", + "format": "date-time" + }, + "resolution": { + "description": "The time span between individual points of information in a time series. Example: 30 s", + "type": [ + "string", + "null" + ], + "title": "Resolution" + }, + "alignment": { + "description": "Indicator whether stamps in a time series are left, right or middle. \"null\" if there is no time series. Example: left", + "type": [ + "string", + "null" + ], + "title": "Alignment" + }, + "aggregationType": { + "description": "Indicates whether the values are a sum, average or current. Example: sum", + "type": [ + "string", + "null" + ], + "title": "Aggregation type" + } + }, + "additionalProperties": false, + "title": "Timeseries" + }, + "title": "Timeseries" + } + }, + "additionalProperties": false, + "title": "Temporal" + }, + "sources": { + "description": "List of source objects. Each object has all name-value-pairs.", + "type": "array", + "items": { + "description": "Source object in list of source objects. Each object has all name-value-pairs.", + "type": "object", + "properties": { + "title": { + "description": "Human readable title of the source, e.g. document title or organisation name. Example: IPCC Fifth Assessment Report", + "type": [ + "string", + "null" + ], + "title": "Title" + }, + "description": { + "description": "Free text description of the data set. Example: Scientific climate change report by the UN", + "type": [ + "string", + "null" + ], + "title": "Description" + }, + "path": { + "description": "URL to original source. Example: https://www.ipcc.ch/site/assets/uploads/2018/02/ipcc_wg3_ar5_full.pdf", + "type": [ + "string", + "null" + ], + "title": "Path", + "format": "uri" + }, + "licenses": { + "description": "The license(s) under which the source(s) is/are provided. List of objects.", + "type": "array", + "items": { + "description": "A license object under which the described source is provided. Each object has all name-value-pairs.", + "type": "object", + "properties": { + "name": { + "description": "SPDX identifier: Example: ODbL-1.0", + "type": [ + "string", + "null" + ], + "title": "Name" + }, + "title": { + "description": "Official (human readable) title. Example: Open Data Commons Open Database License 1.0", + "type": [ + "string", + "null" + ], + "title": "Title" + }, + "path": { + "description": "A link to the license. Example: https://opendatacommons.org/licenses/odbl/1-0/index.html", + "type": [ + "string", + "null" + ], + "title": "Path" + }, + "instruction": { + "description": "Short description of rights and restrictions. Example: You are free to share and change, but you must attribute, and share derivations under the same license.", + "type": [ + "string", + "null" + ], + "title": "Instruction" + }, + "attribution": { + "description": "Copyrightholder of the source. Example: © Intergovernmental Panel on Climate Change 2014", + "type": [ + "string", + "null" + ], + "title": "Attribution" + } + }, + "title": "Licenses" + }, + "title": "Licenses" + } + }, + "additionalProperties": false, + "title": "Sources" + }, + "title": "Sources" + }, + "licenses": { + "description": "The license(s) under which the described package is provided. List of objects.", + "type": "array", + "items": { + "description": "A license object under which the described package is provided. Each object has all name-value-pairs.", + "type": "object", + "properties": { + "name": { + "description": "SPDX identifier. Example: ODbL-1.0", + "type": [ + "string", + "null" + ], + "title": "Name" + }, + "title": { + "description": "Official (human readable) title. Example: Open Data Commons Open Database License 1.0", + "type": [ + "string", + "null" + ], + "title": "Title" + }, + "path": { + "description": "A url-or-path string, that is a fully qualified HTTP address, or a relative POSIX path (see the url-or-path definition in Data Resource for details). Example: https://opendatacommons.org/licenses/odbl/1-0/index.html", + "type": [ + "string", + "null" + ], + "title": "Path" + }, + "instruction": { + "description": "Short description of rights and restrictions. Example: You are free to share and change, but you must attribute, and share derivations under the same license.", + "type": [ + "string", + "null" + ], + "title": "Instruction" + }, + "attribution": { + "description": "Copyrightholder of the produced data set. Example: © Reiner Lemoine Institut", + "type": [ + "string", + "null" + ], + "title": "Attribution" + } + }, + "additionalProperties": false, + "title": "Licenses" + }, + "title": "Licenses" + }, + "contributors": { + "description": "The people or organizations who contributed to this data package. List of objects.", + "type": "array", + "items": { + "description": "A person or organizations who contributed to this data package. Each object refers to one contributor. Every contributor must have a title and property. A path, email, role and organization properties are optional extras.", + "type": "object", + "properties": { + "title": { + "description": "Name/title of the contributor (name for a person, name or title for an organization). Example: Jon Doe", + "type": [ + "string", + "null" + ], + "title": "Title" + }, + "email": { + "description": "E-mail address of the contributor. Example: contact@example.com", + "type": [ + "string", + "null" + ], + "title": "Email", + "format": "email" + }, + "date": { + "description": "Date of the contribution. If the contribution took more than a day, use the date of the final contribiution. Date Format is ISO 8601. Example: 2016-06-16", + "type": [ + "string", + "null" + ], + "title": "Date", + "format": "date" + }, + "object": { + "description": "Target of contribution. Which part of the package was supplied/changed. Example: Metadata", + "type": [ + "string", + "null" + ], + "title": "Object" + }, + "comment": { + "description": "Free text comment on what's been done. Example: Fixed a typo in the title", + "type": [ + "string", + "null" + ], + "title": "Comment" + } + }, + "additionalProperties": false, + "title": "Contributors" + }, + "title": "Contributors" + }, + "resources": { + "description": "Resources, described as a list of data resource format objects.", + "type": "array", + "items": { + "description": "The data resource format describes a data resource as an individual file or table.", + "type": "object", + "properties": { + "profile": { + "description": "A string identifying the profile of this descriptor as per the profiles specification. This information is retained in order to comply with the \"Tabular Data Package\" standard. If at all in doubt the value should read \"tabular-data-resource\". Example: tabular-data-resource", + "type": [ + "string", + "null" + ], + "title": "Profile", + "options": { + "hidden": true + } + }, + "name": { + "description": "A resource MUST contain a name unique to amongst all resources in this data package. To comply with the data package standard it must consist of only lowercase alphanumeric character plus \".\", \"-\" and \"_\". It may not start with a number. In a database this will be the name of the table within its containing schema. It would be usual for the name to correspond to the file name (minus the file-extension) of the data file the resource describes. Example: sandbox.example_table", + "type": [ + "string", + "null" + ], + "title": "Name" + }, + "path": { + "description": "A url-or-path string, that should be a permanent http(s) address or other path directly linking to the resource. Example: directly linking to the resource. https://openenergy-platform.org/dataedit/view/openstreetmap/osm_deu_roads", + "type": [ + "string", + "null" + ], + "title": "Path", + "options": { + "hidden": true + } + }, + "format": { + "description": "\"csv\", \"xls\", \"json\" etc. would be expected to be the standard file extension for this type of resource. When you upload your data to the OEDB, in the shown metadata string, the format will be changed accordingly to \"PostgreSQL\", since the data there are stored in a database. Example: csv", + "type": [ + "string", + "null" + ], + "title": "Format", + "options": { + "hidden": true + } + }, + "encoding": { + "description": "Specifies the character encoding of the resource's data file. The values should be one of the \"Preferred MIME Names\" for a character encoding registered with IANA. If no value for this key is specified then the default is UTF-8. Example: UTF-8", + "type": [ + "string", + "null" + ], + "title": "Encoding", + "options": { + "hidden": true + } + }, + "schema": { + "description": "Object containing fields, primary key and for foreign keys. Describes the structure of the present data.", + "type": "object", + "properties": { + "fields": { + "description": "List of field objects.", + "type": "array", + "items": { + "description": "Field object. Every object describes a column and provides name, description, type and unit.", + "type": "object", + "properties": { + "name": { + "description": "Name string unique within its scope. Example: year", + "type": [ + "string", + "null" + ], + "title": "Name", + "readonly": true + }, + "description": { + "description": "Free-text describing the field. Example: Reference year for which the data were collected.", + "type": [ + "string", + "null" + ], + "title": "Description" + }, + "type": { + "description": "Data type of the field. In case of a geom-column in a database, also indicate the shape and CRS. Example: geometry(Point, 4326)", + "type": [ + "string", + "null" + ], + "title": "Type", + "readonly": true + }, + "isAbout": { + "description": "Ontology URI to describe the column header", + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "description": "Name of the Dataset", + "type": [ + "string", + "null" + ], + "title": "Name" + }, + "path": { + "description": "Path to the OEO (URL)", + "type": [ + "string", + "null" + ], + "title": "Path", + "format": "uri" + } + }, + "additionalProperties": false, + "title": "IsAbout" + }, + "title": "IsAbout" + }, + "valueReference": { + "description": "Ontology URI for an extended description of the values in the column", + "type": "array", + "items": { + "type": "object", + "properties": { + "value": { + "description": "The value this reference is assinged to.", + "type": [ + "string", + "null" + ], + "title": "Value" + }, + "name": { + "description": "Full name of the value", + "type": [ + "string", + "null" + ], + "title": "Name", + "format": "uri" + }, + "path": { + "description": "Path to the OEO", + "type": [ + "string", + "null" + ], + "title": "Path", + "format": "uri" + } + }, + "additionalProperties": false, + "title": "Value reference" + }, + "title": "valueReference" + }, + "unit": { + "description": "Unit, preferably SI-Unit, that values in this field are mapped to. If \"unit\" doesn't apply to a field, use \"null\". Example: MW", + "type": [ + "string", + "null" + ], + "title": "Unit" + } + }, + "additionalProperties": false, + "title": "Field" + }, + "title": "Field" + }, + "primaryKey": { + "description": "A primary key is a field or set of fields that uniquely identifies each row in the table. It's recorded as a list of strings, since it is possible to define the primary key as made up of several columns. Example: id", + "type": "array", + "items": { + "type": [ + "string", + "null" + ], + "title": "Primary key" + }, + "title": "Primary key" + }, + "foreignKeys": { + "description": "List of foreign keys.", + "type": "array", + "items": { + "description": "A foreign key is a field that refers to a column in another table.", + "type": "object", + "properties": { + "fields": { + "description": "The column (as list of items) in the table that is constrainted by the foreign key. Example: version", + "type": "array", + "items": { + "type": [ + "string", + "null" + ], + "title": "Field" + }, + "title": "Fields" + }, + "reference": { + "description": "The reference to the foreign table.", + "type": "object", + "properties": { + "resource": { + "description": "The foreign resource (table). Example: schema.table", + "type": [ + "string", + "null" + ], + "title": "Resource" + }, + "fields": { + "description": "The foreign resource column. List of fields. Example: version", + "type": "array", + "items": { + "type": [ + "string", + "null" + ], + "title": "Field" + }, + "title": "Field" + } + }, + "additionalProperties": false, + "title": "Reference" + } + }, + "additionalProperties": false, + "title": "Foreign Key" + }, + "title": "Foreign Keys" + } + }, + "additionalProperties": false, + "title": "Schema" + }, + "dialect": { + "description": "Object. A CSV Dialect defines a simple format to describe the various dialects of CSV files in a language agnostic manner. In case of a database, the values in the containing fields are \"null\".", + "type": "object", + "properties": { + "delimiter": { + "description": "Specifies the character sequence which should separate fields (aka columns). Common characters are \",\" (comma), \".\" (point) and \"\t\" (tab). Example: ,", + "type": [ + "string", + "null" + ], + "title": "Delimiter" + }, + "decimalSeparator": { + "description": "Symbol used to separate the integer part from the fractional part of a number written in decimal form. Depending on language and region this symbol can be \".\" or \",\". Example: .", + "type": [ + "string", + "null" + ], + "title": "Decimal separator" + } + }, + "additionalProperties": false, + "title": "Dialect", + "options": { + "hidden": true + } + } + }, + "additionalProperties": false, + "title": "Resource" + }, + "title": "Resource" + }, + "review": { + "description": "Data uploaded through the OEP needs to go through review. The review will cover the areas described here: https://github.com/OpenEnergyPlatform/data-preprocessing/wiki and carried out by a team of the platform. The review itself is documented at the specified path and a badge is rewarded with regards to completeness.", + "type": "object", + "properties": { + "path": { + "description": "A URL or path string, that should be a permanent http(s) address directly linking to the documented review. Example: https://www.example.com", + "type": [ + "string", + "null" + ], + "title": "Path" + }, + "badge": { + "description": "A badge of either Bronze, Silver, Gold or Platin is used to label the given metadata based on its quality. Example: Platin", + "type": [ + "string", + "null" + ], + "title": "Badge" + } + }, + "additionalProperties": false, + "title": "Review", + "options": { + "hidden": true + } + }, + "metaMetadata": { + "description": "Object. Description about the metadata themselves, their format, version and license. These fields should already be provided when you’re filling out your metadata.", + "type": "object", + "properties": { + "metadataVersion": { + "description": "Type and version number of the metadata. Example: OEP-1.5", + "type": [ + "string", + "null" + ], + "title": "Metadata version" + }, + "metadataLicense": { + "description": "Object describing the license of the provided metadata.", + "type": "object", + "properties": { + "name": { + "description": "SPDX identifier. Example: CC0-1.0", + "type": [ + "string", + "null" + ], + "title": "Name" + }, + "title": { + "description": "Official (human readable) license title. Example: Creative Commons Zero v1.0 Universal", + "type": [ + "string", + "null" + ], + "title": "Title" + }, + "path": { + "description": "Url or path string, that is a fully qualified HTTP address. Example: https://creativecommons.org/publicdomain/zero/1.0/", + "type": [ + "string", + "null" + ], + "title": "Path" + } + }, + "additionalProperties": false, + "title": "Metadata license" + } + }, + "additionalProperties": false, + "title": "Meta metadata", + "options": { + "hidden": true + } + }, + "_comment": { + "description": "Object. The “_comment”-section is used as a self-description of the final metadata-file. It is text, intended for humans and can include a link to the metadata documentation(s), required value formats and similar remarks. The comment section has no fix structure or mandatory values, but a useful self-description, similar to the one depicted here, is encouraged.", + "type": "object", + "properties": { + "metadata": { + "description": "Reference to the metadata documentation in use. Example: Metadata documentation and explanation (https://github.com/OpenEnergyPlatform/organisation/wiki/metadata)", + "type": [ + "string", + "null" + ], + "title": "Metadata" + }, + "dates": { + "description": "Comment on data/time format. Example: Dates and time must follow the ISO8601 including time zone (YYYY-MM-DD or YYYY-MM-DDThh:mm:ss±hh)", + "type": [ + "string", + "null" + ], + "title": "Dates" + }, + "units": { + "description": "Comment on units. Example: If you must use units in cells (which is discouraged), leave a space between numbers and units (100 m)", + "type": [ + "string", + "null" + ], + "title": "Units" + }, + "languages": { + "description": "Comment on language format. Example: Languages must follow the IETF (BCP47) format (en-GB, en-US, de-DE)", + "type": [ + "string", + "null" + ], + "title": "Languages" + }, + "licenses": { + "description": "Reference to license format. Example: License name must follow the SPDX License List (https://spdx.org/licenses/)", + "type": [ + "string", + "null" + ], + "title": "Licenses" + }, + "review": { + "description": "Reference to review documentation. Example: Following the OEP Data Review (https://github.com/OpenEnergyPlatform/data-preprocessing/wiki)", + "type": [ + "string", + "null" + ], + "title": "Review" + }, + "null": { + "description": "Feel free to add more descriptive comments. Like \"null\". Example: If a field is not applicable just enter \"null\"", + "type": [ + "string", + "null" + ], + "title": "Null" + }, + "todo": { + "description": "If an applicable value is not yet available and will be inserted later on use: 'todo' ", + "type": [ + "string", + "null" + ], + "title": "Todo" + } + }, + "title": "_comment" + } + }, + "additionalProperties": false +} \ No newline at end of file diff --git a/metadata/v151/schema.py b/metadata/v151/schema.py new file mode 100644 index 00000000..2af91a92 --- /dev/null +++ b/metadata/v151/schema.py @@ -0,0 +1,5 @@ +import json +import os + +with open(os.path.join(os.path.dirname(__file__), "schema.json"), "rb") as f: + OEMETADATA_LATEST_SCHEMA = json.loads(f.read()) diff --git a/metadata/v151/template.json b/metadata/v151/template.json new file mode 100644 index 00000000..670ff605 --- /dev/null +++ b/metadata/v151/template.json @@ -0,0 +1,196 @@ +{ + "name": null, + "title": null, + "id": null, + "description": null, + "language": [ + null + ], + "subject": [ + { + "name": null, + "path": null + } + ], + "keywords": [ + null + ], + "publicationDate": null, + "context": { + "homepage": null, + "documentation": null, + "sourceCode": null, + "contact": null, + "grantNo": null, + "fundingAgency": null, + "fundingAgencyLogo": null, + "publisherLogo": null + }, + "spatial": { + "location": null, + "extent": null, + "resolution": null + }, + "temporal": { + "referenceDate": null, + "timeseries": [ + { + "start": null, + "end": null, + "resolution": null, + "alignment": null, + "aggregationType": null + }, + { + "start": null, + "end": null, + "resolution": null, + "alignment": null, + "aggregationType": null + } + ] + }, + "sources": [ + { + "title": null, + "description": null, + "path": null, + "licenses": [ + { + "name": null, + "title": null, + "path": null, + "instruction": null, + "attribution": null + } + ] + }, + { + "title": null, + "description": null, + "path": null, + "licenses": [ + { + "name": null, + "title": null, + "path": null, + "instruction": null, + "attribution": null + } + ] + } + ], + "licenses": [ + { + "name": null, + "title": null, + "path": null, + "instruction": null, + "attribution": null + } + ], + "contributors": [ + { + "title": null, + "email": null, + "date": null, + "object": null, + "comment": null + } + ], + "resources": [ + { + "profile": null, + "name": null, + "path": null, + "format": null, + "encoding": null, + "schema": { + "fields": [ + { + "name": null, + "description": null, + "type": null, + "unit": null, + "isAbout": [ + { + "name": null, + "path": null + } + ], + "valueReference": [ + { + "value": null, + "name": null, + "path": null + } + ] + }, + { + "name": null, + "description": null, + "type": null, + "unit": null, + "isAbout": [ + { + "name": null, + "path": null + } + ], + "valueReference": [ + { + "value": null, + "name": null, + "path": null + } + ] + } + ], + "primaryKey": [ + null + ], + "foreignKeys": [ + { + "fields": [ + null + ], + "reference": { + "resource": null, + "fields": [ + null + ] + } + } + ] + }, + "dialect": { + "delimiter": null, + "decimalSeparator": "." + } + } + ], + "@id": null, + "@context": null, + "review": { + "path": null, + "badge": null + }, + "metaMetadata": { + "metadataVersion": "OEP-1.5.0", + "metadataLicense": { + "name": "CC0-1.0", + "title": "Creative Commons Zero v1.0 Universal", + "path": "https://creativecommons.org/publicdomain/zero/1.0/" + } + }, + "_comment": { + "metadata": "Metadata documentation and explanation (https://github.com/OpenEnergyPlatform/oemetadata)", + "dates": "Dates and time must follow the ISO8601 including time zone (YYYY-MM-DD or YYYY-MM-DDThh:mm:ss±hh)", + "units": "Use a space between numbers and units (100 m)", + "languages": "Languages must follow the IETF (BCP47) format (en-GB, en-US, de-DE)", + "licenses": "License name must follow the SPDX License List (https://spdx.org/licenses/)", + "review": "Following the OEP Data Review (https://github.com/OpenEnergyPlatform/data-preprocessing/blob/master/data-review/manual/review_manual.md)", + "null": "If not applicable use: null", + "todo": "If a value is not yet available, use: todo" + } +} \ No newline at end of file diff --git a/metadata/v151/template.py b/metadata/v151/template.py new file mode 100644 index 00000000..78a3a705 --- /dev/null +++ b/metadata/v151/template.py @@ -0,0 +1,5 @@ +import json +import os + +with open(os.path.join(os.path.dirname(__file__), "template.json"), "rb") as f: + OEMETADATA_LATEST_TEMPLATE = json.loads(f.read()) From de77aafd456063e946e94f418a4a34eec9bd1570 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Wed, 23 Feb 2022 11:49:53 +0100 Subject: [PATCH 07/11] add and update test for v1.5.1 --- tests/metadata/v151/__init__.py | 0 tests/metadata/v151/test_example.py | 10 ++++++++++ tests/metadata/v151/test_schema.py | 28 ++++++++++++++++++++++++++++ tests/metadata/v151/test_template.py | 10 ++++++++++ 4 files changed, 48 insertions(+) create mode 100644 tests/metadata/v151/__init__.py create mode 100644 tests/metadata/v151/test_example.py create mode 100644 tests/metadata/v151/test_schema.py create mode 100644 tests/metadata/v151/test_template.py diff --git a/tests/metadata/v151/__init__.py b/tests/metadata/v151/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/metadata/v151/test_example.py b/tests/metadata/v151/test_example.py new file mode 100644 index 00000000..b5d2343a --- /dev/null +++ b/tests/metadata/v151/test_example.py @@ -0,0 +1,10 @@ +def test_if_example_json_loads_successfully(): + from metadata.v151.example import OEMETADATA_V151_EXAMPLE + + +def test_example_against_schema_which_should_succeed(): + import jsonschema + from metadata.v151.example import OEMETADATA_V151_EXAMPLE + from metadata.v151.schema import OEMETADATA_V151_SCHEMA + + assert jsonschema.validate(OEMETADATA_V151_EXAMPLE, OEMETADATA_V151_SCHEMA) == None diff --git a/tests/metadata/v151/test_schema.py b/tests/metadata/v151/test_schema.py new file mode 100644 index 00000000..ea6c488f --- /dev/null +++ b/tests/metadata/v151/test_schema.py @@ -0,0 +1,28 @@ +def test_if_schema_json_loads_successfully(): + try: + from metadata.v151.schema import OEMETADATA_V151_SCHEMA + except Warning: + print("Metadata Schema v1.5.1 cant load. Check if the files are missing!") + + +def test_if_schema_json_has_correct_schema_and_id_set(): + from metadata.v151.schema import OEMETADATA_V151_SCHEMA + import string + + def get_string(s): + return string.printable + s + string.printable + + assert get_string(OEMETADATA_V151_SCHEMA["$schema"]) == get_string("http://json-schema.org/draft-07/schema#") + + # https://raw.githubusercontent.com/OpenEnergyPlatform/oemetadata/master/oemetadata/v150/schema.json + assert get_string(OEMETADATA_V151_SCHEMA["$id"]) == get_string( + "https://raw.githubusercontent.com/OpenEnergyPlatform/oemetadata/master/oemetadata/v151/schema.json" + ) + + +def test_schema_against_metaschema_which_should_succeed(): + import jsonschema + from metadata.v151.schema import OEMETADATA_V151_SCHEMA + from metadata.metaschema.draft07.schema import OEMETADATA_METASCHEMA_DRAFT07_SCHEMA + + assert jsonschema.validate(OEMETADATA_V151_SCHEMA, OEMETADATA_METASCHEMA_DRAFT07_SCHEMA) is None diff --git a/tests/metadata/v151/test_template.py b/tests/metadata/v151/test_template.py new file mode 100644 index 00000000..5140030a --- /dev/null +++ b/tests/metadata/v151/test_template.py @@ -0,0 +1,10 @@ +def test_if_template_json_loads_successfully(): + from metadata.v151.template import OEMETADATA_V151_TEMPLATE + + +def test_template_against_schema_which_should_succeed(): + import jsonschema + from metadata.v151.template import OEMETADATA_V151_TEMPLATE + from metadata.v151.schema import OEMETADATA_V151_SCHEMA + + assert jsonschema.validate(OEMETADATA_V151_TEMPLATE, OEMETADATA_V151_SCHEMA) == None From 1e151cf927294bc8b1fc13d73186aaedb296b76b Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Wed, 23 Feb 2022 11:53:57 +0100 Subject: [PATCH 08/11] fix module versions --- metadata/v151/example.py | 2 +- metadata/v151/schema.py | 2 +- metadata/v151/template.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/metadata/v151/example.py b/metadata/v151/example.py index 1d30161e..5058e7a9 100644 --- a/metadata/v151/example.py +++ b/metadata/v151/example.py @@ -2,4 +2,4 @@ import os with open(os.path.join(os.path.dirname(__file__), "example.json"), "rb") as f: - OEMETADATA_LATEST_EXAMPLE = json.loads(f.read()) + OEMETADATA_V151_EXAMPLE = json.loads(f.read()) diff --git a/metadata/v151/schema.py b/metadata/v151/schema.py index 2af91a92..749ba41e 100644 --- a/metadata/v151/schema.py +++ b/metadata/v151/schema.py @@ -2,4 +2,4 @@ import os with open(os.path.join(os.path.dirname(__file__), "schema.json"), "rb") as f: - OEMETADATA_LATEST_SCHEMA = json.loads(f.read()) + OEMETADATA_V151_SCHEMA = json.loads(f.read()) diff --git a/metadata/v151/template.py b/metadata/v151/template.py index 78a3a705..16da240a 100644 --- a/metadata/v151/template.py +++ b/metadata/v151/template.py @@ -2,4 +2,4 @@ import os with open(os.path.join(os.path.dirname(__file__), "template.json"), "rb") as f: - OEMETADATA_LATEST_TEMPLATE = json.loads(f.read()) + OEMETADATA_V151_TEMPLATE = json.loads(f.read()) From 58214a3dd7ef868e2ed511919467b9525f93da95 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Wed, 23 Feb 2022 11:57:55 +0100 Subject: [PATCH 09/11] fix id --- metadata/v151/schema.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/metadata/v151/schema.json b/metadata/v151/schema.json index 49ebcaaf..8cc3d65b 100644 --- a/metadata/v151/schema.json +++ b/metadata/v151/schema.json @@ -1,6 +1,6 @@ { "$schema": "http://json-schema.org/draft-07/schema#", - "$id": "https://raw.githubusercontent.com/OpenEnergyPlatform/oemetadata/master/oemetadata/latest/schema.json", + "$id": "https://raw.githubusercontent.com/OpenEnergyPlatform/oemetadata/master/oemetadata/v151/schema.json", "description": "Open Energy Plaftorm (OEP) metadata schema latest", "type": "object", "properties": { From 62bda0cbe8d9df8a64f903b32d30ac94c1f3685e Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Wed, 23 Feb 2022 11:59:49 +0100 Subject: [PATCH 10/11] update changelog --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f3c586f7..06fe78a2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,6 +39,8 @@ Here is a template for new release sections - Use [GitHub projects](https://github.com/OpenEnergyPlatform/oemetadata/projects) to organise releases - Add new example table to show connection to OEO - Add test for latest directory to CI [PR#74] +- Add tests for v151 [PR#81] +- Add release version directory for oem-v151 ### Changed - Update ``subject`` to work with OEO From 4b20943fc3cf8c95c4eca899b0c6e80246f87bd5 Mon Sep 17 00:00:00 2001 From: jh-RLI Date: Wed, 23 Feb 2022 12:06:06 +0100 Subject: [PATCH 11/11] update release procedure --- RELEASE_PROCEDURE.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/RELEASE_PROCEDURE.md b/RELEASE_PROCEDURE.md index 1f83636f..eb3fbeb4 100644 --- a/RELEASE_PROCEDURE.md +++ b/RELEASE_PROCEDURE.md @@ -5,6 +5,14 @@ Status of all issues and PR's assigned to the project. If there are any open issues or PR's, decide if you want to include them them to be included in the upcoming release. Make sure that all changes are are included in the development branch before proceeding to the next steps. +- Create a new folder named "vXXX" (release version) in the metadata and +test directory. + - In the metadata direcotry copy all files from latest to the + new folder then adapt all .py files to the latest version and + update the id in schema.json. + - In test copy all files from previous oder latest version folder + in test directory and adapt all tests (test_***.py files) to + the latest version. ## Release - 1 Checkout a new branch from devolop branch named: release/oem-vX.X.X - 2 Update the following files: