[alpha] Initial release

This release adds initial metrics and model components as:

1) Metrics

We can import various metrics from evalem.metrics

• BasicMetrics and SemanticMetrics can be used
• basic metrics are:
  - F1Metric
  - RecallMetric
  - PrecisionMetric
  - ConfusionMatrix
  - AccuracyMetric
  - ExactMatchMetric
• semantic metrics include BertScore and BartScore

These metrics can be used independently to evaluate the predictions from upstream models using references/ground-truths.

See PRs this, this and this

2) ModelWrapper

evalem.models include various model wrapper implementation. See PRs this and this

• evalem.models.QuestionAnsweringHFPipelineWrapper and evalem.models.TextClassificationHFPipelineWrapper are now the main wrappers for QA and Text Classification tasks respectively.

  • These also have better parameter initialization, allowing any suitable models and tokenizers to be used along with device types.
  • hf_params dict is also provided as a parameter that will be used for initializing the HF pipeline

• The model wrappers utilize 2 distinct processing parameters (one for pre-preocessing and one for post-processing) which should be Callable (lambda function, external modules that can be called, etc.) and can be modified accordingly to pre/post processing.

3) Evaluator

evaluators provide abstraction/containerization of metrics to evaluate in group.
See PRs this, this and this

We have 2 different evaluator implementation:

• evalem.evaluators.QAEvaluator for evaluating QA metrics
• evalem.evaluators.TextClassificationEvaluator for text classification

We can also directly use evalem.evaluators._base.Evaluator to create our own custom evaluator object.