Tools and utilities for supporting AIDA evaluations.
| Name | Description | Location | Latest release branch |
|---|---|---|---|
| AIDA-Evaluation Docker | Tool for applying the evaluation pipeline to task1/2/3 submissions | tools/aida-evaluation | AIDAED-v2022.1.1 |
| Name | Description | Location | Latest release branch |
|---|---|---|---|
| AIDA-Evaluation Docker | Tool for applying the entire evaluation pipeline to task1 (or the pre-pooling steps to task2 and task3) submissions | tools/aida-evaluation | AIDAED-v2020.3.0 |
| Name | Description | Location | Latest release branch |
|---|---|---|---|
| SPAQRL Query Application Docker | The tool for applying SPARQL queries to KBs | tools/sparql-query-application | AIDAQA-v2019.2.5 |
| AIDA-ValidateResponses | Tool for validating responses obtained after applying SPARQL queries to KBs | tools/validate-responses | AIDAVR-v2019.0.3 |
| AIDA-ConfidenceAggregation | Tool (and docker) for aggregating confidences | tools/confidence-aggregation | AIDACA-v2019.0.2 |
| AIDA-ScoreResponses | Tool for scoring responses | tools/scorer | AIDASR-v2019.2.1 |
| AIF Generator | Tool for converting annotations into AIF | python/generate_aif.py | AIDAAG-v2019.0.1 |
| AIDA-Evaluation Docker | Tool for applying the entire evaluation pipeline | tools/aida-evaluation | AIDAED-v2019.0.1 |
| Name | Description |
|---|---|
| utils/translate-annotations/seedling | Utility to translate LDC's seedling annotation to turtle-RDF format |
| tools/resolve-queries | Tool for applying SPARQL queries to a knowledge base in AIF format in order to produce output in the simplified xml response format |
| tools/validate-responses | Tool for validating responses for AIDA M9 evaluation |
Please refer to tool/utility specific README file(s) for more details.