Releases: microsoft/CameraTraps
Pytorch-Wildlife v1.1.0
🎉🎉🎉 Pytorch-Wildlife Version 1.1.0
MegaDetectorV6 is finally out
After a few months of public beta testing, we are finally ready to officially release our 6th version of MegaDetector, MegaDetectorV6! In the next generation of MegaDetector, we are focusing on computational efficiency, performance, mordernizing of model architectures, and licensing. We have trained multiple new models using different model architectures, including Yolo-v9, Yolo-v11, and RT-Detr for maximum user flexibility. We have a rolling release schedule for different versions of MegaDetectorV6, and in the first step, we are releasing the compact version of MegaDetectorV6 with Yolo-v9 (MDv6-ultralytics-yolov9-compact, MDv6-c in short). From now on, we encourage our users to use MegaDetectorV6 as their default animal detection model.
This MDv6-c model is optimized for performance and low-budget devices. It has only one-sixth (SMALLER) of the parameters of the previous MegaDetectorV5 and exhibits 12% higher recall (BETTER) on animal detection in our validation datasets. In other words, MDv6-c has significantly fewer false negatives when detecting animals, making it a more robust animal detection model than MegaDetectorV5. Furthermore, one of our testers reported that the speed of MDv6-c is at least 5 times FASTER than MegaDetectorV5 on their datasets.
| Models | Parameters | Precision | Recall |
|---|---|---|---|
| MegaDetectorV5 | 121M | 0.96 | 0.73 |
| MegaDetectroV6-c | 22M | 0.92 | 0.85 |
Learn how to use MegaDetectorV6 in our image demo and video demo.
HerdNet in Pytorch-Wildlife
We have incorporated a point-based overhead animal detection model into our model zoo called HerdNet (Delplanque et al. 2022). Two model weights are incorporated in this release, HerdNet-general (their default weights) and HerdNet-ennedi (their model trained on Ennedi 2019 datasets). More details can be found here and in their original repo. This is the first third-party model in Pytorch-Wildlife and the foundation of our expansion to overhead/aerial animal detection and classification. Please see our HerdNet demo on how to use it!
Customed Weights
You can now load custom weights you fine-tuned on your own datasets using the finetuning module directly in the Pytorch-Wildlife pipeline! Please see the demo on how to do it. You can also load it in our Gradio app!
Folder Separation
You can now automatically separate your image detections into folders based on detection results! Please see our folder separation demo on how to do it. You can also test it in our Gradio demo!
Simplified Batch Detection!
We have also simplified the batch detection pipeline. Now we do not need to define pytorch datasets and dataloaders specifically.
Previous:
tgt_folder_path = os.path.join(".","demo_data","imgs")
dataset = pw_data.DetectionImageFolder(
tgt_folder_path,
transform=pw_trans.MegaDetector_v5_Transform(target_size=detection_model.IMAGE_SIZE,
stride=detection_model.STRIDE),
extension='JPG'
loader = DataLoader(dataset, batch_size=32, shuffle=False,
pin_memory=True, num_workers=0, drop_last=False)
results = detection_model.batch_image_detection(loader)Now:
tgt_folder_path = os.path.join(".","demo_data","imgs")
results = detection_model.batch_image_detection(tgt_folder_path, batch_size=16)Fixes and other updates:
Pytorch-Wildlife v1.0.2.4
Hello everyone! We have just updated Pytorch-Wildlife to version 1.0.2.4. This is a small update but fixes some bugs related to result outputs reported in issue #428 . Besides the bug fixes, we have also added a HuggingFace demo and have two google colab demos Image_detection_demo and Video_detection_demo. We also have fully revamped our README for better visual and created a discord server for our community. Please join us! Thank you so much!
Pytorch-Wildlife v1.0.2
We have just updated Pytorch-Wildlife to version 1.0.2. Although this is a relatively minor release in terms of the package itself, it is a big update with the addition of a module many have requested: the classification fine-tuning module.
In the Pytorch-Wildlife package, we've introduced an additional default classification model trained on the Snapshot Serengeti camera trap dataset. This is the first camera trap classification model we've trained for the African region, in collaboration with Snapshot Safari under Zooniverse. Furthermore, Pytorch-Wildlife is now citable using BibTeX. We are also working on a brief technical report to allow Pytorch-Wildlife to be cited as a paper.
For the classification fine-tuning module, we are currently making it independent of the base package to preserve code-base clarity. This module provides functions that utilize the MegaDetector under Pytorch-Wildlife to pre-crop input camera trap images. It includes data splitting functions based on random split, location split, and shooting sequence split, alongside the entire classification training pipeline. This module produces classification weights that can be directly linked back to other Pytorch-Wildlife functionalities, such as the Gradio demo app. Currently, the classification fine-tuning module is based on ResNet. However, users are more than welcome to employ any classification architecture they prefer, provided some adjustments are made to adhere to the separate feature encoder + classifier convention required by the Pytorch-Wildlife framework. We chose this convention to simplify future utilities and analyses, such as embedding space visualization.
Please do not hesitate to submit any issues you may encounter, enhancement PRs that you think may be good additions, and let us know what you think!
Thank you!
Pytorch-Wildlife v1.0.1
In this updated version, we've addressed some memory leak problems that originated from utilizing numpy within Pytorch's dataset and dataloader. The primary cause was the letterbox function, which we've now completely overhauled. Initially, this function, implemented by Ultralytics, relied entirely on numpy. However, numpy is known to conflict with Pytorch's dataset and dataloader, as documented in this issue (pytorch/pytorch#13246). To circumvent this problem, we've re-implemented the function using only PIL and Pytorch.
Alongside this, we've also modified some post-processing functions to prepare our codebase for dockerization. Currently, we're in the process of launching a Docker image with Pytorch-Wildlife to simplify environment management. We appreciate your patience during this time!
Pytorch-Wildlife v1.0
Welcom to Pytorch-Wildlife v1.0
What is Pytorch-Wildife
The Pytorch-Wildlife library is a new AI Conservation platform that allows users to directly load the MegaDetector v5 model weights for animal detection. We've fully refactored our codebase, prioritizing ease of use in model deployment and expansion. In addition to MegaDetector v5, Pytorch-Wildlife also accommodates a range of classification weights, such as those derived from an Amazon Rainforest camera trap dataset and an Opossum classification dataset (more details of these datasets will be published soon).
Unlike previous implementations where the MegaDetector v5 model weights need to be manually downloaded, Pytorch-Wildlife automatically downloads all the model weights (including MegaDetector v5 and other classification model weights) the first time the models are loaded. This implementation uses an approach similar to the default Pytorch model loading method.
Please also explore the codebase and functionalities of Pytorch-Wildlife through our interactive Gradio interface and detailed Jupyter notebooks, designed to showcase the practical applications of our enhancements.
Pytorch-Wildlife can be installed through:
pip install PytorchWildlife
More information can be found in our Installation guide and detailed library documentation.
Here is a brief example on how to perform detection and classification on a single image using PyTorch-wildlife:
import torch
from PytorchWildlife.models import detection as pw_detection
from PytorchWildlife.models import classification as pw_classification
img = torch.randn((3, 1280, 1280))
# Detection
detection_model = pw_detection.MegaDetectorV5()
detection_result = detection_model.single_image_detection(img)
#Classification
classification_model = pw_classification.AI4GAmazonRainforest()
classification_results = classification_model.single_image_classification(img)MegaDetector(s) 5.0
What's new?
This release incorporates additional training data, specifically aiming to improve our coverage of:
- Boats and trains in the "vehicle" class
- Artificial objects (e.g. bait stations, traps, lures) that frequently overlap with animals
- Rodents, particularly at close range
- Reptiles and small birds
This release also represents a change in MegaDetector's architecture, from Faster-RCNN to YOLOv5. Our inference scripts have been updated to support both architectures, so the transition should be mostly seamless. Users will need to update to the latest version of the repo and follow our updated setup instructions in order to use MDv5, but the inference scripts work the same way they always have.
Why are there two files?
MDv5 is actually two models (MDv5a and MDv5b), differing only in their training data (see the training data section on the MegaDetector page for details). Both appear to be more accurate than MDv4, and both are 3x-4x faster (!!!) than MDv4, but each MDv5 model can outperform the other slightly, depending on your data. Guidelines about which to start with are TBD; we will work with the community to develop these guidelines over the next few months. When in doubt, for now, try them both. If you really twist our arms to recommend one... we recommend MDv5a. But try them both and tell us which works better for you!
A word on confidence thresholds
MDv5 uses the full range of confidence values much more than MDv4 does, so don't apply the MDv4 thresholds you're accustomed to when using MDv5 results. Typical confidence thresholds for MDv4 were in the 0.7-0.8 range; for MDv5, they are in the 0.15-0.25 range. Does that make MDv5 better-calibrated? Who knows, but what we know for sure is that if you apply a confidence threshold like 0.8 to MDv5 results, you'll miss some animals.
Will MDv5 files work with Timelapse?
Short answer: yes!
Long answer...
Many users read MegaDetector results into Timelapse. MDv4 and MDv5 results generated with our batch inference script are compatible with old versions of Timelapse, but as per above, MDv4 and MDv5 operate in difference confidence ranges, and the default confidence thresholds in old versions of Timelapse were set to be reasonable for MDv4. We want users to have reasonable default values regardless of which model version you're using, so we've updated the MegaDetector output format to include recommended defaults; as of the just-released Timelapse version 2.2.5.2, Timelapse reads those defaults, so users will see reasonable default thresholds for either MDv4 or MDv5 files. You should be prompted to update Timelapse when you next start it, or you can visit the Timelapse download page to get the latest version. If you are using Timelapse version 2.2.5.1 or earlier and for any reason you can't upgrade, just remember to bump up those thresholds if you're working with MDv5 results.
But of course, as always, never trust default confidence values; always spend some time choosing thresholds that are appropriate for your data!
And remember to thank your friendly neighborhood Timelapse developer (Saul) for all the work he put in to supporting this release.
Is it really more accurate and faster than MDv4? That seems too good to be true.
Well, it's definitely faster, that part is easy.
Out of what we think is a healthy paranoia, we've spent a bunch of time over the last few weeks trying to find cases where MDv5 is less accurate than MDv4. Although we have found a few individual images where MDv4 finds something that MDv5 misses (because deep learning can be maddening that way), on every data set we've tried with any realistic number of images, we've failed to find cases where MDv4 is the more accurate option, and on many data sets that challenged MDv4, MDv5 is substantially more accurate. But we're sure that if we keep looking, we will find a case where MDv4 is better for some esoteric reason, because... see above, re: deep learning being maddening.
So as much as we want to hear about your successes with MDv5, we're also eager to hear about the cases where it fails, or isn't as good as MDv4. Email us to let us know how things compare for you, or file issues on this repo, or post your experiences to the AI for Conservation forum at WILDLABS or the AI for Conservation Slack group.
What if I'm still cautious and/or skeptical? Can I compare MDv4 and MDv5 results?
Although we wouldn't exactly call this "user-friendly" yet, in the course of our own comparisons on mostly-unlabeled data, we put together a script to compare sets of batch results, which we've primarily used to compare MDv4/MDv5a/MDv5b. This produces pages like this one, which asks "what does each model find at a reasonable confidence threshold that the other models miss?". If all models are equal, each model will have around the same number of detections that the others don't have, and they will all be junk / false detections. Hopefully this script is helpful to others who will be doing similar comparisons.
(Take that particular results link with a grain of salt; it's a very practical data set for demonstrating this comparison script, but the data it contains is from the ENA24 data set, which is included in MDv5's training data, and not MDv4's.)
Along the same lines, in some cases, users may want to run more than one model, and take, e.g., only the very-high-confidence detections from MDv5b (or MDv4) and merge them into an otherwise-MDv5a results file. We have a new script to do this as well, although we think this will rarely be necessary.
Tell us how it goes!
We are really excited about this release... but really what we're even more excited about is hearing from users about how it can help your work. So please email us to let us know how MDv5 works for you, or file issues on this repo. No detail is too small to let us know about, especially as it pertains to the updated setup/execution instructions. If you're confused, someone else it too.
And if you want to discuss your MDv5 experiences with community, consider posting to the AI for Conservation forum at WILDLABS or the AI for Conservation Slack group.
We look forward to hearing from you!
MegaDetector v4.1, now with a "vehicle" class!
This version of the MegaDetector incorporates additional training data from Borneo, Australia, other island habitats, and a bunch of locations in the WCS Camera Traps dataset, amongst others. We also added images with humans in both daytime and nighttime to make the "person" class more robust. And as promised, we debut a "vehicle" class for cars, trucks, tractors, and bicycles.
Here's MDv4.1 on a (old?) picture from the WCS set:
Proportion of the three classes in our training set:
To download this version's model configuration file, checkpoint files and saved model for TF Serving, visit the MegaDetector page.
Don't ask about v4.0 :) Hope everyone's staying safe during COVID.