Bolt is a light-weight library for deep learning. Bolt, as a universal deployment tool for all kinds of neural networks, aims to minimize the inference runtime as much as possible. Higher speed, better security and more efficient memory management are the advantages that Bolt strives to provide. Bolt has been widely deployed and used in many departments of HUAWEI company, such as 2012 Laboratory, CBG and HUAWEI Product Lines. Feel free to make good use of issue submission, or join our QQ chatroom (Chinese): 833345709.
Generally, there are two steps to get started with bolt. It's quiet easy for users to quickly running bolt.
-
Conversion: use X2bolt to convert your model from caffe,onnx,tflite or tensorflow to .bolt;
-
Inference: run benchmark with .bolt and data to get the inference result.
For more details about the usage of X2bolt and benchmark tools, see docs/USER_HANDBOOK.md.
-
caffe, onnx, tflite, tensorflow
-
Float32, Float16, Int8, 1-bit
-
ARM CPU(v7, v8, v8.2), Mali GPU, X86(AVX2)
-
Bolt has shown its high performance in the inference of common CV and NLP neural networks. Some of the representative networks that we have verified are listed below. You can find detailed benchmark information in docs/BENCHMARK.md.
Application Models CV Squeezenet/Mobilenet_v1/Mobilenet_v2/Mobilenet_v3/Resnet50
/Ghostnet/SSD/Yolov3/Pointnet/...etc.NLP Bert/TinyBert/Albert/Neural Machine Translation/Text To Speech
/Automatic Speech Recognition/...etc.More DL Tasks ... More models than these mentioned above are supported, users are encouraged to further explore.
-
- Graph Optimization
- Thread Affinity
- Algorithm Tuning
- Time-Series Data Acceleration
Everything you want to know about bolt is recorded in the detailed documentations stored in docs.
- How to install bolt with different compilers.
- How to use bolt to inference your ML models.
- How to develop bolt to customize more models.
- Benchmark results on some universal models.
- How to build APP with kit.
- Frequently Asked Questions(FAQ)
Bolt refers to the following projects: caffe, onnx, tensorflow, ncnn, mnn, dabnn.
The MIT License(MIT)