This repository is the official implementation of Energy-based Out-of-distribution Detection by Weitang Liu, Xiaoyun Wang, John Owens and Yixuan Li. This method is an effective and easy OOD detector with and without fine-tuning. Our code is implemented with courtesy of Outlier-Exposure. If you have any code related questions, such as this issue and this issue, we highly recommened to check the couterpart in Outlier-Exposure.
Pretrained models are provided in folder
./CIFAR/snapshots/
Please download the datasets in folder
./data/
run energy score testing for cifar10 WRN
bash run.sh energy 0
run energy score testing for cifar100 WRN
bash run.sh energy 1
run energy score training and testing for cifar10 WRN
bash run.sh energy_ft 0
run energy score training and testing for cifar100 WRN
bash run.sh energy_ft 1
Our model achieves the following average performance on 6 OOD datasets:
1. MSP vs energy score with and without fine-tuned on CIFAR-10
| Model name | FPR95 |
|---|---|
| Softmax score | 51.04% |
| Energy score (ours) | 33.01% |
| Softmax score with fine-tune | 8.53% |
| Energy score with fine-tune (ours) | 3.32% |
3. Performance among different baselines for WideResNet
CIFAR-10:
| Model name | FPR95 |
|---|---|
| Softmax score | 51.04% |
| Energy score (ours) | 33.01% |
| ODIN | 35.71% |
| Mahalanobis | 37.08% |
| Outlier Exposure | 8.53% |
| Energy score with fine-tune (ours) | 3.32% |
CIFAR-100:
| Model name | FPR95 |
|---|---|
| Softmax score | 80.41% |
| Energy score (ours) | 73.60% |
| ODIN | 74.64% |
| Mahalanobis | 54.64% |
| Outlier Exposure | 58.10% |
| Energy score with fine-tune (ours) | 47.55% |
These experiments make use of numerous outlier datasets. Links for less common datasets are as follows, 80 Million Tiny Images Textures, Places365, LSUN-C, LSUN-R, iSUN and SVHN.
@article{liu2020energy,
title={Energy-based Out-of-distribution Detection},
author={Liu, Weitang and Wang, Xiaoyun and Owens, John and Li, Yixuan},
journal={Advances in Neural Information Processing Systems},
year={2020}
}