A proposal for assessing the FAIRness of data in Trusted Digital Repositories

[etsoupra]

(see the FAIR Data Principles)

1. Blog post by Emily Thomas on Data Impact Blog by UKDS
2. EUDAT Webinar by Peter Doorn and Ingrid Dillo
3. GO FAIR Metrics Group

[CaroleGoble]

Are the FAIR Data Principles fair?
Dunning, Alastair; de Smaele, Madeleine; Böhmer, Jasmin
https://zenodo.org/record/321423#.WX9Do1GQyvE

[etsoupra]

Thanks a lot for your comment :)
Very interesting article indeed - we are exploring how the information included can help us to make further improvements on the tool we are working on.

[dr-shorthair]

Nice. I particularly like this emphasis:
"the facets provide targets that will help ..."

[sjDCC]

Thanks for the references @etsoupra - really useful. I've seen Peter and Ingrid's webinar, but the information in your post and details of the GO-FAIR metrics group will help as we write this up.

I've just been playing around with the assessment tool and had a couple of questions about the scores applied in the Survey Routing Diagrams:

• Will all data get a score of 1 or above? If you follow the 'no' paths (e.g. no PID, no metadata, no user licence, proprietary formats etc) it seems the dataset will get a score of 1 at each stage. Did you decide against 0 scores?

• Are you considering applying the scores or weightings to metadata as well as data? The main difference in the accessibility criteria is whether the data is open, public access so I guess some sensitive data will never be able to get a high FAIR rating, even if all the metadata are accessible and machine-readable etc. I read the Accessibility criteria A1.2 (the protocol allows for an authentication and authorization procedure, where necessary) as allowing closed or restricted access datasets. It might be useful to find a path that allows for non-open data to score as highly e.g. by awarding scores for clear metadata and info on how to gain access.

• What kind of response have you been getting to rebranding the final R as a resultant score? Is this working well so far or do you think you'll change the approach in the GO-FAIR metrics group?

Thanks again for all the inputs. We'll definitely be including this in the report 👍

[etsoupra]

Dear Sarah, Thank you very much for your email and feedback. With regards to your questions: * Yes, it is correct that the dataset will always score 1 or above at each stage. We decided not to include 0 scores because we wanted to create criteria for each star level (i.e. 1, 2, 3…) as we had already created 5 levels of criteria. Therefore, to add a 0 component would add an additional 6th criteria, for which we did not see necessary, considering we had already the 5 levels. We also thought it was perhaps a little more motivational to receive one star even when a dataset has nothing compared to receiving a 0 score! So, in short, yes all datasets will always score 1 or above. * We have recently received some comments considering this point from some external reviews, so are currently in the process of updating the tool based on this feedback. One of the main points which came up was this issue about sensitive data receiving a lower score just because there is restricted access, therefore we are considering a higher star level for this kind of data if they have rich metadata. Additionally, if the restricted dataset allows users to request access then it may be able to score higher. * Our operationalisation seems to work well so far, however, we are in the process of testing and making adjustments and improvements on the tool so that may be something we will consider. Thanks again for the comments. Looking forward to the report ☺ Best, Eleftheria Tsoupra & Emily Thomas From: Sarah Jones <notifications@github.com> Reply-To: FAIR-Data-EG/consultation <reply@reply.github.com> Date: Sunday, 20 August 2017 at 14:39 To: FAIR-Data-EG/consultation <consultation@noreply.github.com> Cc: Eleftheria Tsoupra <eleftheria.tsoupra@dans.knaw.nl>, Mention <mention@noreply.github.com> Subject: Re: [FAIR-Data-EG/consultation] A proposal for assessing the FAIRness of data in Trusted Digital Repositories (#23) Thanks for the references @etsoupra<https://github.com/etsoupra> - really useful. I've seen Peter and Ingrid's webinar, but the information in your post and details of the GO-FAIR metrics group will help as we write this up. I've just been playing around with the assessment tool and had a couple of questions about the scores applied in the Survey Routing Diagrams<https://github.com/FAIR-Data-EG/consultation/files/1237051/SurveyRoutingDiagrams.pdf>: * Will all data get a score of 1 or above? If you follow the 'no' paths (e.g. no PID, no metadata, no user licence, proprietary formats etc) it seems the dataset will get a score of 1 at each stage. Did you decide against 0 scores? * Are you considering applying the scores or weightings to metadata as well as data? The main difference in the accessibility criteria is whether the data is open, public access so I guess some sensitive data will never be able to get a high FAIR rating, even if all the metadata are accessible and machine-readable etc. I read the Accessibility criteria A1.2 (the protocol allows for an authentication and authorization procedure, where necessary) as allowing closed or restricted access datasets. It might be useful to find a path that allows for non-open data to score as highly e.g. by awarding scores for clear metadata and info on how to gain access. * What kind of response have you been getting to rebranding the final R as a resultant score? Is this working well so far or do you think you'll change the approach in the GO-FAIR metrics group? Thanks again for all the inputs. We'll definitely be including this in the report 👍 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#23 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AZeeEQnP3FcNIEbOHlNXxM1KxmA89WEKks5saCkEgaJpZM4ONdNj>.