This repo uses RetinaNet for object detection in remote sensing imagery (here, seals in a drone-derived orthomosaic), and includes a set of tools for using geospatial data with CNNs. The scripts have been generalized to accommodate other uses beyond seals with custom inputs. All operations are currently written to be executed in Google Colab, though they can be easily adapted to run on a local machine. To run in Colab, if you are viewing in GitHub or locally in Jupyter Notebook, click the "launch in Google Colab" button at the top to launch the script in Google Colab instead. If you clone or fork this project be sure to update those pointer buttons in the markdown to link to your new version.
You will need:
(1) a web browser
(2) a google drive account (you can create and use a "dummy" account if you want)
(3) a file unzipper (such as 7-Zip, WinRAR or WinZip)
Each script pulls in files from a Google Drive folder, which you must specify, if using your own data, and spits out files that will download in your browser, usually as a zip archive that you must extract. When you move on to the next script if using your own data you must drop these files into a publicly shared Google Drive folder and point the script to that Google folder. It doesn't have to be the same drive folder every time, but makes sense to do it that way because some files get used multiple times in the workflow.
Pay attention to the markdown at the start of each script, which specifies what's needed to run the script. The markdown at the end of each script explains what's ahead, and how the outputs fit together in the next steps.
Note that each of these scripts will prompt you to allow Google permissions to use Google Cloud SDK. You will need to grant permissions and paste a code into an input box at the start of each script when pulling files from Google Drive. After that, the code should run smoothly without requiring any further input (though some variables require customization if using your own dataset).
Estimated time for each step:
Step 1: split a large image or mosaic into tiles
< 5 minutes + time to download mosaic tiles (varies by mosaic size)
Step 2: create training data, if using your own
Multiple hours (varies depending on size of mosiac, number of annotations, ease of detecting annotations, familiarity with VIA)
Step 3: split tiles and associated training data into training, validation and testing subsets
< 1 minute
Step 4: train and then test a CNN model
< 10 minutes if using a small "step" count (ranges much longer by default, depending on size of dataset)
Step 5: clean and export the results as shapefiles
< 1 minute
You'll also spend a little time handling files between steps.
To start this workflow with your own files, create a google drive folder (it does not need to be your personal google account, you may want to create a new account to store these data). Create a folder on that drive and upload the orthomosaic you're working with (the default folder/mosaic is https://drive.google.com/drive/folders/1wuAONrdYYNylyb_ZV2hpd-SrYjk-UXty). Make sure the drive is shared publicly ("anyone with the link can view") and be sure to point to your working folder's link in each script.
You can demo the workflow on our grey seal dataset by using the default files, accessible via the default folder pointers.
Note: workflow is designed for working on one mosaic at a time, but inputs and outputs can be merged/separated between steps to process multi-mosaic datasets.
If you encounter any errors or have any questions or suggestions, please reach out to Gregory Larsen at gdl10@duke.edu
Code was written by Patrick Gray, Maddie Hayes and Gregory Larsen with support from the Duke University Marine Robotics and Remote Sensing Lab.