Skip to content

Latest commit

 

History

History
100 lines (78 loc) · 4.43 KB

README.md

File metadata and controls

100 lines (78 loc) · 4.43 KB
Raw

DyRep: Bootstrapping Training with Dynamic Re-parameterization

Official implementation for paper "DyRep: Bootstrapping Training with Dynamic Re-parameterization", CVPR 2022.

By Tao Huang, Shan You, Bohan Zhang, Yuxuan Du, Fei Wang, Chen Qian, Chang Xu.

🔥 Training code is available here.

DyRep Framework

Updates

March 11, 2022

The code is available at image_classification_sota.

Getting started

Clone training code

git clone https://github.com/hunto/DyRep.git --recurse-submodules
cd DyRep/image_classification_sota

Then prepare your environment and datasets following the README.md in image_classification_sota.

Implementation of DyRep

The core concept of DyRep is in lib/models/utils/dyrep.py.

Reproducing our results

CIFAR

Dataset Model Config Paper This repo Log
CIFAR-10 VGG-16 config 95.22% 95.37% log
CIFAR-100 VGG-16 config 74.37% 74.60% log
  • CIFAR-10 
    sh tools/dist_train.sh 1 configs/strategies/DyRep/cifar.yaml nas_model --model-config configs/models/VGG/vgg16_cifar10.yaml --dyrep --experiment dyrep_cifar10_vgg16
  • CIFAR-100 
    sh tools/dist_train.sh 1 configs/strategies/DyRep/cifar.yaml nas_model --model-config configs/models/VGG/vgg16_cifar100.yaml --dyrep --dataset cifar100 --experiment dyrep_cifar100_vgg16

ImageNet

Dataset Model Config Paper This repo Log
ImageNet ResNet-18 config 71.58% 71.66% log
ImageNet ResNet-50 config 77.08% 77.22% log

  • ResNets

    sh tools/dist_train.sh 8 configs/strategies/DyRep/resnet.yaml resnet18 --dyrep --experiment dyrep_imagenet_res18

  • MobileNetV1

    sh tools/dist_train.sh 8 configs/strategies/DyRep/mbv1.yaml mobilenet_v1 --dyrep --experiment dyrep_imagenet_mbv1

  • RepVGG

    • DyRep-A2 
      sh tools/dist_train.sh 8 configs/strategies/DyRep/repvgg_baseline.yaml timm_repvgg_a2 --dyrep --dyrep_recal_bn_every_epoch --experiment dyrep_imagenet_repvgg_a2
    • DyRep-B2g4 and DyRep-B3 
      sh tools/dist_train.sh 8 configs/strategies/DyRep/repvgg_strong.yaml timm_repvgg_b2g4 --dyrep --dyrep_recal_bn_every_epoch --experiment dyrep_imagenet_repvgg_b2g4

Deploying the Trained DyRep Models to Inference Models

sh tools/dist_run.sh tools/convert.py ${GPUS} ${CONFIG} ${MODEL} --resume ${CHECKPOINT}

For example, if you want to deploy the trained ResNet-50 model with the best checkpoint, run

sh tools/dist_run.sh tools/convert.py 8 configs/strategies/DyRep/resnet.yaml resnet50 --dyrep --resume experiments/dyrep_imagenet_res50/best.pth.tar

Then it will run test before and after deployment to ensure the accuracy will not drop.

The final weights of the inference model will be saved in experiments/dyrep_imagenet_res50/convert/model.ckpt.

License

This project is released under the Apache 2.0 license.

Citation

@InProceedings{Huang_2022_CVPR,
    author    = {Huang, Tao and You, Shan and Zhang, Bohan and Du, Yuxuan and Wang, Fei and Qian, Chen and Xu, Chang},
    title     = {DyRep: Bootstrapping Training With Dynamic Re-Parameterization},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {588-597}
}