Create json schemas describing both data models

[areleu]

Overview

Hello, I have been working on integrating this data model without friction into frictionless. This requires effort in different repositories Here are the pull requests and issues:

Frictionless:
frictionlessdata/frictionless-py#1229
frictionlessdata/frictionless-py#1228

oemetadata:

OpenEnergyPlatform/oemetadata#91

If these changes take place then validating the models will work directly with frictionless. I think the next step is being able to validate the structure of the tables themselves, to do this we would need descriptors in json format.

Task

Write a data descriptors for the data model. These would comprise of the json schema representation of the packages:

I can contribute to this next month but if someone starts this that would be great.

[jh-RLI]

@areleu
I have updated the json schema in the oemetadata repo. There were some wrong entries :/. Now the validation is running successfully. Is this what you meant by this issue?
Regarding OpenEnergyPlatform/oemetadata#91, I will also leave this open for now, but I think it is no longer necessary (did I understand that correctly?)

[areleu]

@areleu I have updated the json schema in the oemetadata repo. There were some wrong entries :/. Now the validation is running successfully. Is this what you meant by this issue? Regarding OpenEnergyPlatform/oemetadata#91, I will also leave this open for now, but I think it is no longer necessary (did I understand that correctly?)

i think OpenEnergyPlatform/oemetadata#91 is no longer relevant. I think is better if we have a schema specific to the packages. I Started to write one but I am not done. Let me upload what I have as a github gist

[areleu]

@areleu I have updated the json schema in the oemetadata repo. There were some wrong entries :/. Now the validation is running successfully. Is this what you meant by this issue? Regarding OpenEnergyPlatform/oemetadata#91, I will also leave this open for now, but I think it is no longer necessary (did I understand that correctly?)

It is correct, i think OpenEnergyPlatform/oemetadata#91 is no longer relevant. I think is better if we have a schema specific to the packages. I Started to write one but I am not done. Let me upload what I have as a github gist

https://gist.github.com/areleu/6fe09323f26ec3ed4d83bc9084ab7ea9

As I said, this is not yet completely functional.

[areleu]

The justification for having such a thing is that. You can validate if the metadata complies with the OEMetadata but you can't really check if it is a valid OEdatamodel implementation. For doing so one would need to write a schema that extends the OEMetadata to check for the particular constraints of the OEDatamodel.

This way you can validate very easily if an user is filling their tables correctly.

If we have these schemas, I think we can even use frictionless for validation out of the box.

[jh-RLI]

@henhuy you mentioned some updates to the oedatamodel. @areleu is working on a json schema that can be used with frionless to validate data. Maybe it is useful to share the changes so we can also create a schema for the updated version.

[jh-RLI]

@henhuy I think you have already implemented this in the oeadatamodel api?

[henhuy]

Yes. I implemented an API for our project, which loads metadata from existing OEP table, creates frictionless schema from OEMetadata schema and validates incoming data from CSV file via frictionless resource. If everything is okay, data gets uploaded otherwise a frictionless error report is returned to user.
Here the corresponding code for loading and validating data via OEMetadata (you have to look into upload module)