Development of Fast Refinement Detectors on AI Edge Platforms

Under review

Abstract

Refinement detector (RefineDet) is a state-of-the-art model in object detection that has been developed and refined based on high-end GPU systems. In this study, we discovered that the speed of models developed in high-end GPU systems is inconsistent with that in embedded systems. In other words, the fastest model that operates on high-end GPU systems may not be the fastest model on embedded boards. To determine the reason for this phenomenon, we performed several experiments on RefineDet using various backbone architectures on three different platforms: NVIDIA Titan XP GPU system, Drive PX2 board, and Jetson Xavier board. Finally, we achieved real-time performances (approximately 20 fps) based on the experiments on AI edge platforms such as NVIDIA Drive PX2 and Jetson Xavier boards. We believe that our current study would serve as a good reference for developers who wish to apply object detection algorithms to AI edge computing hardwares. The complete code and models are publicly available on the web (link).

1. Installation

1.1 Requirement

• Ubuntu 16.04
• Caffe
• Python2
• CUDA 9.0

1.2 ReifineDet

• Install original RefineDet source from git (link)
  git clone https://github.com/sfzhang15/RefineDet.git

• Fllowing step to train and test with original RefineDet using COCO dataset git (link)

1.3 Rebuild for Modified RefineDet

• Download requirements from this repository
  git clone https://github.com/mkchoi-0323/modified_refinedet.git

• Rebuild caffe for original RefineDet with new caffe layers (depthwise convolution and axpy laer)
  cd RefineDet
  make clean

  1. copy every .cpp and .cu files in addtional_layers
  2. paste .cpp and .cu files to original RefineDet/src/caffe/layers/
  3. copy every .hpp files in additional_layers
  4. paste .hpp files to original RefineDet/include/caffe/layers/

• Rebuild by caffe compile command
  make all -j4

2. Training

• Download desired python code to generate head and backbone

  • For example, you should prepare MobileNetV1_COCO_320.py and model_libs_MobileNetV1.py to train new model of RefineDet with MobileNetV1 blocks

• Make model path and copy python trainin interface
  mkdir RefineDet/models/MobileNet/coco/refinedet_mobilenetv1_320x320
  cp modified_refinedet/heads/MobileNetV1_COCO_320.py RefineDet/models/MobileNet/coco/refinedet_mobilenetv1_320x320/

• Download pre-trained backbone (link)

• Copy pre-trained backbone to model path
  cp modified_refinedet/pretrained_backbones/mobielnet.caffemodel RefineDet/models/MobileNet/

• Copy model interface to original RefineDet
  mv RefineDet/python/caffe/model_libs.py RefineDet/python/caffe/model_libs.py.backup
  cp modified_refinedet/heads/model_libs_MobileNetV1.py RefineDet/python/caffe/model_libs.py

• Run training interface
  cd RefineDet
  python models/MobileNet/coco/refinedet_mobilenetv1_320x320/MobileNetV1_COCO_320.py

3. Testing

• Copy dataset loader to original RefineDet interface
  mv RefineDet/test/lib/dataset/coco.py RefineDet/test/lib/dataset/coco.py.backup
  mv RefineDet/test/lib/dataset/factory.py RefineDet/test/lib/dataset/factory.py.backup
  mv modified_refinedet/test/lib/dataset/coco.py RefineDet/test/lib/dataset/
  mv modified_refinedet/test/lib/dataset/factory.py RefineDet/test/lib/dataset/

• Copy testing interface
  mv modified_refinedet/test/lib/dataset/coco.py RefineDet/test/lib/dataset/
  mv modified_refinedet/refinedet_test_MobileNet.py

• Download pre-trained models (link)

• Set model and deploy path to proper location

3.1 val2017

• Set model and deploy path to proper location
  vim modified_refinedet/test/refinedet_test_MobileNet.py

• Run test
  cd RefineDet
  python modified_refinedet/test/refinedet_test_MobileNet.py

3.2 test-dev2017

• Set model and deploy path to proper location
  vim modified_refinedet/test/refinedet_test-dev.py

• Run test
  cd RefineDet
  python modified_refinedet/test/refinedet_test-dev.py