Releases: deepjavalibrary/djl
Releases · deepjavalibrary/djl
DJL v0.13.0 release
DJL v0.13.0 brings the new TensorRT and Python engines, and updates the engines PyTorch to 1.9.0, ONNXRuntime to 1.9.0, PaddlePaddle to 2.0.2, and introduces several new features:
Key Features
- Introduces TensorRT engine
- Introduces Python engine which allows you to run Python scripts with DJL
- Upgrades PyTorch engine to 1.9.0 with CUDA 11.1 support
- Upgrades ONNXRuntime engine to 1.9.0 with UINT8 support
- Upgrades PaddlePaddle engine to 2.0.2
- Introduces the djl-bench snap package:
sudo snap install djlbench --classic - Introduces dynamic batch feature for djl-serving (#1154)
- DJL serving becomes a standalone repository: https://github.com/deepjavalibrary/djl-serving (#1170)
- Allows load ModelZoo model using url (#1120)
- Support .npy and .npz file format (#1131)
- djl-serving is available in dockerhub
- Publishes d2l book Chinese translation preview (chapter 1-5) : https://d2l-zh.djl.ai
Enhancement
- Introduces several new features in djl-serving:
- Improves djl-serving API to make it easy to get HTTP headers (#1134)
- Loads models on all GPUs at startup for djl-serving (#1132)
- Enables asynchronous logging for djl-serving
- Makes djl-serving access log in separate log file (#1150)
- Adds configuration to support number worker threads for GPU inference (#1153)
- Improves auto-scale algorithm for djl-serving (#1149)
- Introduces several new features in djl-bench:
- Adds a command line option to djl-bench to generate NDList file (#1155)
- Adds warmup to benchmark (#1152)
- Improves djll-bench to support djl:// urls (#1146)
- Adds support to benchmark on multiple GPUs (#1144)
- Adds support to benchmark onnx on GPU machines (#1148)
- Adds support to benchmark TensorRT models (#1257)
- Adds support to benchmark Python models (#1267)
- Introduces several new features in PyTorch engine:
- Supports PyTorch custom input data type with IValue (#1208)
- Introduces several new features in OnnxRuntime:
- Adds UINT8 support for OnnxRuntime (#1271)
- Introduces several new features in PaddlePaddle:
- Introduces several API improvements:
- Adds missing NDList.get(String) API (#1194)
- Adds support to directly load models from a TFHub url (#1231)
- Improves repository API to support passing argument in the URL query string (#1139)
- Avoids loading the default engine if it is not being used (#1136)
- Improves IO by adding a buffer to read/write (#1135)
- Improves NDArray.toString() debug mode performance (#1142)
- Makes GPU device detection engine specific to avoid confusion when using multiple engines (#1138)
Documentation and examples
- Adds Style Transfer example with CycleGAN (#1180)
Breaking change
- Removes support for Apache MXNet 1.6.0
- Deprecates Device.getDevices() API - Use Engine.getDevices() instead
- Renames SimpleVocabulary to DefaultVocabulary
Bug Fixes
- Fixes broken link in documents
- Fixes TensorFlow NDArray was created on CPU instead of GPU bug (#1279)
- Fixes default image processing pipeline (#1268)
- Fixed XGBoost NDArray multiple read bug (#1239)
- Fixes platform matching bug (#1167)
- Fixes NullPointerException in NDArray.toString() (#1157)
- Fixes PaddlePaddle crash due to GC (#1162)
- Fixes NDArrayAdapter.getSparseFormat() unsupported bug (#1151)
- Fixes mixed device issue in multiple engine use case (#1123)
- Fixes handle duplicate plugin issue for djl-serving (#1108)
- Fixes XGBoost NDArray creation bug (#1109)
- Fixes runtime exception running benchmark arm machine(#1107)
- Fixes unregister model regression (#1101)
Contributors
This release is thanks to the following contributors:
- Akshay Rajvanshi (@aksrajvanshi)
- Aziz Zayed (@AzizZayed)
- Elchanan Haas (@ElchananHaas)
- Erik Bamberg (@ebamberg)
- Frank Liu (@frankfliu)
- Jake Lee (@stu1130)
- Kimi MA (@kimim)
- Paul Greyson
- Qing Lan (@lanking520)
- Raymond Liu (@raymondkhliu)
- Sindhu Somasundaram (@sindhuvahinis)
- Zach Kimberg (@zachgk)
DJL v0.12.0 release
DJL v0.12.0 added GPU support to PaddlePaddle and ONNXRuntime, and introduces several new features:
Key Features
- Updates PaddlePaddle engine with GPU support.
- Updates ONNXRuntime engine with GPU support.
- Upgrades ONNXRuntime engine to 1.8.0.
- Upgrades XGBoost engine to 1.4.1.
- Introduces AWS Inferentia support, see our example for detail.
- Adds FLOAT16 datatype support in NDArray.
- Support UTF16 surrogate characters in NLP tokenization.
- Makes benchmark as a standalone tool.
- Releases djl-serving docker image to docker hub.
Enhancement
- DJL Benchmark now can benchmark any datatype as input.
- Makes Grayscale image processing match openCV’s behavior (#965)
- Improves PyTorch engine to load extra shared library for custom operators (#983)
- Improves djl-serving REST API to support load model on specified engine (#977)
- Improves djl-serving to support load multiple version of a model on the same endpoint (#1052)
- Improves djl-serving to support auto-scale workers based on traffic (#986)
- Implements several operators:
- Introduces several API improvements
Documentation and examples
- Adds Low cost inference with AWS Inferentia demo.
- Adds BigGAN demo in examples (#1038)
- Adds Super-resolution demo in examples (#1049)
Breaking change
- Direct access ModelZoo ModelLoader is no longer supported, use Criteria API instead.
- Deprecates ModelZoo.loadModel() API in favor of using Criteria.loadModel().
Bug Fixes
- Fixes missing softmax in action_recognition model zoo model (#969)
- Fixes saveModel NPE bug (#989)
- Fixes NPE bug in block.toString() function (#1076)
- Adds back String tensor support to TensorFlow engine (lost in 0.11.0 during refactor) (#1040)
- Sets ai.djl.pytorch.num_interop_threads default value for djl-serving (#1059)
Known issues
- The TensorFlow engine has a known memory leak issue due to the JavaCPP dependency. The memory leak issue has been fixed in javacpp 1.5.6-SNAPSHOT. You have to manually include javacpp 1.5.6-SNAPSHOT to avoid the memory leak. See: https://github.com/deepjavalibrary/djl/tree/master/tensorflow/tensorflow-engine#installation for more details.
Contributors
This release is thanks to the following contributors:
- Akshay Rajvanshi(@aksrajvanshi (https://github.com/ghost))
- Aziz Zayed(@AzizZayed (https://github.com/AzizZayed))
- Erik Bamberg(@ebamberg (https://github.com/ebamberg))
- Frank Liu(@frankfliu (https://github.com/frankfliu))
- Hodovo(@hodovo (https://github.com/Hodovo))
- Jake Lee(@stu1130 (https://github.com/stu1130))
- Qing Lan(@lanking520 (https://github.com/lanking520))
- Tibor Mezei (@zemei (https://github.com/zemei))
- Zach Kimberg(@zachgk (https://github.com/zachgk))
DJL v0.11.0 release note
DJL v0.11.0 brings the new engines XGBoost 1.3.1, updates PyTorch to 1.8.1, TensorFlow to 2.4.1, Apache MXNet 1.8.0, PaddlePaddle to 2.0.2 and introduces several new features:
Key Features
- Supports XGBoost 1.3.1 engine inference: now you can run prediction using models trained in XGBoost.
- Upgrades PyTorch to 1.8.1 with CUDA 11.1 support.
- Upgrades TensorFlow to 2.4.1 with CUDA 11.0 support.
- Upgrades Apache MXNet to 1.8.0 with CUDA 11.0 support.
- Upgrades PaddlePaddle to 2.0.2.
- Upgrades SentencePiece to 0.1.95.
- Introduces the djl-serving brew package: now you can install djl-serving with
brew install djl-serving. - Introduces the djl-serving plugins.
- Introduces Amazon Elastic Inference support.
Enhancement
- Improves TensorFlow performance by reducing GC and fixed memory leaking issue (#892)
- djl-serving now can run all the engines out-of-box (#886)
- Improves DJL training by using multi-threading on each GPU (#743)
- Implements several operators:
- Adds setGraphExecutorOptimize option for PyTorch engine. (#904)
- Introduces String tensor support for ONNXRuntime (#724)
- Introduces several API improvements
- Introduces model searching feature in djl central (#799)
Documentation and examples
- Introduces DJL tutorials - How to load model on DJL Youtube Channel
- Adds the PaddlePaddle load model documentation (#811)
- Adds the documentations for profiler (#722)
- Adds face detection and face recognition examples (#814)
- Adds model training visualization demo using Vue
Breaking change
- Renames CheckpointsTrainingListener to SaveModelTrainingListener (#686)
- Removes erroneous random forest application (#726)
- Deletes DataManager class (#691)
- Classes under ai.djl.basicdataset packages has been moved into each sub-packages.
Bug Fixes
- Fixes BufferOverflowException when handling handling subimage (#866)
- Fixes ONNXRuntime 2nd engine dependency from IrisTranslator (#853)
- Fixes sequenceMask error when n dimension is 2 (#828)
- Fixes TCP port range buf in djl-serving (#773)
- Fixes one array case for concat operator (#739)
- Fixes non-zero operator for PyTorch (#704)
Known issues
- TensorFlow engine has known memory leak issue due to JavaCPP dependency. The memory leak issue has been fixed in javacpp 1.5.6-SNAPSHOT. User has to manually include javacpp 1.5.6-SNAPSHOT to avoid memory leak. See: https://github.com/deepjavalibrary/djl/tree/master/tensorflow/tensorflow-engine#installation for more detail.
Contributors
This release is thanks to the following contributors:
- Akshay Rajvanshi(@aksrajvanshi (https://github.com/ghost))
- Anthony Feenster(@anfee1 (https://github.com/anfee1))
- Calvin(@mymagicpower (https://github.com/mymagicpower))
- enpasos(@enpasos (https://github.com/enpasos))
- Erik Bamberg(@ebamberg (https://github.com/ebamberg))
- Frank Liu(@frankfliu (https://github.com/frankfliu))
- G Goswami(@goswamig (https://github.com/goswamig))
- Hugo Miguel Ferreira(@hmf (https://github.com/hmf))
- Hodovo(@hodovo (https://github.com/Hodovo))
- Jake Lee(@stu1130 (https://github.com/stu1130))
- Lai Wei(@roywei (https://github.com/roywei))
- Qing Lan(@lanking520 (https://github.com/lanking520))
- Marcos(@markbookk (https://github.com/markbookk))
- Stan Kirdey(@skirdey (https://github.com/skirdey))
- Zach Kimberg(@zachgk (https://github.com/zachgk))
- 付颖志 (@fuyz (https://github.com/fuyz))
- 石晓伟(@Shixiaowei02 (https://github.com/Shixiaowei02))
DJL v0.10.0 Release
DJL v0.10.0 brings the new engines PaddlePaddle 2.0 and TFLite 2.4.1, updates PyTorch to 1.7.1, and introduces several new features:
Key Features
-
Supports PaddlePaddle 2.0 engine inference: now you can run prediction using models trained in PaddlePaddle.
-
Introduces the PaddlePaddle Model Zoo with new models. Please see examples for how to run them.
-
Upgrades TFLite engine to v2.4.1. You can convert TensorFlow SavedModel to TFLite using this converter.
-
Introduces DJL Central to easily browse and view models available in DJL’s ModelZoo.
-
Introduces generic Bert Model in DJL (#105)
-
Upgrades PyTorch to 1.7.1
Enhancement
- Enables listing input and output classes in ModelZoo lookup (#624)
- Improves PyTorch performance by using PyTorch index over engine agnostic solution (#638)
- Introduces various fixes and improvements for MultiThreadedBenchmark (#617)
- Makes the default engine deterministic when multiple engines are in dependencies(#603)
- Adds norm operator (similar to numpy.linalg.norm.html) (#579)
- Refactors the DJL Trackers to use builder patterns (#562)
- Adds the NDArray stopGradient and scaleGradient functions (#548)
- Model Serving now supports scaling up and down (#510)
Documentation and examples
- Introduces DJL 101 Video Series on DJL Youtube Channel
- Adds the documentation for Applications (#673)
- Adds an introduction for Engines (#660)
- Adds documents for DJL community and forums (#646)
- Adds documents on community leaders (#572)
- Adds more purpose to the block tutorial and miscellaneous docs (#607)
- Adds a DJL Paddle OCR Example (#568)
- Adds a TensorFlow amazon review Jupyter notebook example
Breaking change
- Renames DJL-Easy to DJL-Zero (#519)
- Makes RNN operators generic across engines (#554)
- Renames CheckpointsTrainingListener to SaveModelTrainingListener (#573)
- Makes Initialization optional in training (#533)
- Makes
SoftmaxCrossEntropyLoss'sfromLogitflag mean inputs are un-normalized (#639) - Refactors the Vocabulary builder API
- Refactors the SymbolBlock with AbstractSymbolBlock (#491)
Bug Fixes
- Fixes the benchmark rss value #656
- Fixes the recurrent block memory leak and the output shape calculation (#556)
- Fixes the NDArray slice size (#550)
- Fixes #493: verify-java plugin charset bug (#496)
- Fixes #484: support arbitrary URL scheme for repository
- Fixes AbstractBlock inputNames and the inputShapes mismatch bug
Known issues
- The training tests fail on GPU and Windows CPU if 3 engines(MXNet, PyTorch, TensorFlow) are loaded and run together
Contributors
This release is thanks to the following contributors:
- Akshay Rajvanshi(@aksrajvanshi )
- Anthony Feenster(@anfee1)
- Christoph Henkelmann(@chenkelmann)
- Frank Liu(@frankfliu)
- G Goswami(@goswamig)
- Jake Lee(@stu1130)
- James Zow(@Jzow)
- Lai Wei(@roywei)
- Marcos(@markbookk)
- Qing Lan(@lanking520)
- Tibor Mezei(@zemei)
- Zach Kimberg(@zachgk)
- Erik Bamberg(@ebamberg)
- Keerthan Vasist(@keerthanvasist)
- mpskowron(@mpskowron)
DJL v0.8.0 release note
DJL 0.8.0 is a release closely following 0.7.0 to fix a few key bugs along with some new features.
Key Features
- Search model zoo with criteria
- Standard BERT transformer and WordpieceTokenizer for more BERT tasks
- Simplify MRL and Remove Anchor
- Simplify and Standardize CV Model
- Improve Model describe input and output
- String NDArray support (only for TensorFlow Engine)
- Add erfinv operator support
- MXNet 1.6.0 backward compatibility, now you can switch MXNet versions (1.6 and 1.7) using DJL 0.8.0
- Combined pytorch-engine-precxx-11 and pytorch-engine package
- Upgrade onnx runtime from 1.3.1 to 1.4.0
Documentation and examples
- Object Detection with TensorFlow saved model example
- Text Classification with TensorFlow BERT model example
- Added more documentation on TensorFlow engine.
Bug Fixes
- Fixed MXNet multithreading bug and updated multi-threading documentation
- Fixed TensorFlow 2.3 native binaries for Windows platform
Known issues
- You need to add your own Translator when loading image classification models(ResNet, MobileNet) from TensorFlow model Zoo, refer to the example here.
Contributors
Thank you to the following community members for contributing to this release:
Dennis Kieselhorst, Frank Liu, Jake Cheng-Che Lee, Lai Wei, Qing Lan, Zach Kimberg, uniquetrij
DJL v0.6.0 release notes
DJL 0.6.0 brings stable Android support, ONNX Runtime experimental inference support, experimental training support for PyTorch.
Key Features
- Stable Android inference support for PyTorch models
- Provide abstraction for Image processing using ImageFactory
- Experimental support for inference on ONNX models
- Initial experimental training and imperative inference support for PyTorch engine
- Experimental support for using multi-engine
- Improved usage for NDIndex - support for ellipsis notation, arguments
- Improvements to AbstractBlock to simplify custom block creation
- Added new datasets
Documentation and examples
- Added Sentiment Analysis training example
- Updates the DJL webpage with new demos
Breaking changes
- ModelZoo Configuration changes
- ImageFactory changes
- Please refer to javadocs for minor API changes
Known issues
- Issue with training with MXNet in multi-gpu instances
Contributors
Thank you to the following community members for contributing to this release:
Christoph Henkelmann, Frank Liu, Jake Lee, JonTanS, Keerthan Vasist, Lai Wei, Qing, Qing Lan, Victor Zhu, Zach Kimberg, ai4java, aksrajvanshi
DJL v0.5.0 release notes
DJL 0.5.0 release brings TensorFlow engine inference, initial NLP support and experimental Android inference with PyTorch engine.
Key Features
- TensorFlow engine support with TensorFlow 2.1.0
- Support NDArray operations, TensorFlow model zoo, multi-threaded inference
- PyTorch engine improvement with PyTorch 1.5.0
- Experimental Android Support with PyTorch engine
- MXNet engine improvement with MXNet 1.7.0
- Initial NLP support with MXNet engine
- Training LSTM models
- Support various text/word embedding, Seq2Seq use cases
- Added NLP datasets
- New AWS-AI toolkit to integrate with AWS technologies
- Load model from s3 buckets directly
- Improved model-zoo with more models
- Check out new models in Basic Model Zoo, MXNet Model Zoo, PyTorch Model Zoo, TensorFlow Model Zoo
Documentation and examples
- Checkout our new java doc site with version support
- New tutorials and examples
- New demos in our djl-demo repository
- Checkout our latest blog posts:
Breaking changes
- We moved our repository module under api module. There will be no 0.5.0 version for
ai.djl.repository, useai.djl.apiinstead. - Please refer to DJL Java Doc for some minor API changes.
Know issues:
- Issue when using multiple Engines at the same time: #57
- Issue using DJL with Quarkus: #67
- We saw random crash on mac for transfer Learning on CIFAR-10 Dataset example on Jupyter Notebook. Command line all works.
DJL v0.4.1 release notes
DJL 0.4.1 release includes an important performance Improvement on MXNet engine:
Performance Improvement:
- Cached MXNet features. This will avoid
MxNDManager.newSubManager()to repeatedly callinggetFeature()which will make JNA calls to native code.
Known Issues:
Same as v0.4.0 release:
- PyTorch engine doesn't fully support multithreaded inference. You may see random crashes. Single-threaded inference is not impacted. We expect to fix this issue in a future release.
- We saw random crash on mac for “transfer Learning on CIFAR-10 Dataset” example on Jupyter Notebook. Command line all works.
DJL v0.4.0 release notes
DJL 0.4.0 brings PyTorch and TensorFlow 2.0 inference support. Now you can use these engines directly from DJL with minimum code changes.
Note: TensorFlow 2.0 currently is in PoC stage, users will have to build from source to use it. We expect TF Engine finish in the future releases.
Key Features
- Training improvement
- Add InputStreamTranslator
- Model Zoo improvement
- Add LocalZooProvider
- Add ListModels API
- PyTorch Engine support
- Use the new ai.djl.pytorch:pytorch-native-auto dependency for automatic engine selection and a simpler build/installation process
- 60+ methods supported
- PyTorch ModelZoo support
- Image Classification models: ResNet18 and ResNet50
- Object Detection model: SSD_ResNet50
- TensorFlow 2.0 Engine support
- Support on Eager Execution for imperative mode
- 30+ methods support
- TensorFlow ModelZoo support
- Image Classification models: ResNet50, MobileNetV2
Breaking Changes
There are a few changes in API and ModelZoo packages to adapt to multi-engine support. Please follow our latest examples to update your code base from 0.3.0 to 0.4.0.
Known Issues
- PyTorch engine doesn't fully support multithreaded inference. You may see random crashes. Single-threaded inference is not impacted. We expect to fix this issue in a future release.
- We saw random crash on mac for “transfer Learning on CIFAR-10 Dataset” example on Jupyter Notebook. Command line all works.
DJL v0.3.0 release notes
This is the v0.3.0 release of DJL
Key Features
- Use the new
ai.djl.mxnet:mxnet-native-autodependency for automatic engine selection and a simpler build/installation process - New Jupyter Notebook based tutorial for DJL
- New Engine Support for:
- FastText Engine
- Started implementation on a PyTorch Engine
- Simplified training experience featuring:
- TrainingListeners to easily provide full featured training
- DefaultTrainingConfig now contains a default optimizer and initializer
- Easier to transfer from examples to your own code
- Specify the random seed for reproducible training
- Run with multiple engines and specify the default using the "DJL_DEFAULT_ENGINE" environment variable or "ai.djl.default_engine" system property
- Updated ModelZoo design to support unified loading with Criteria
- Simple random Hyperparameter optimization
Breaking Changes
DJL is working to further improve the ease of use and correctness of our API. To that end, we have made a number of breaking changes for this release. Here are a few of the areas that had breaking changes:
- Renamed TrainingMetrics to Evaluator
- CompositeLoss replaced with AbstractCompositeLoss and SimpleCompositeLoss
- Modified MLP class
- Remove Matrix class
- Updates to NDArray class
Known Issues
- RNN operators do not working with GPU on Windows.
- Only CUDA_ARCH 37, 70 are supported for Windows GPU machine.