Segment Anything Model 2 CPP Wrapper for macOS and Ubuntu CPU/GPU

This code is to run a Segment Anything Model 2 ONNX model in c++ code and implemented on the macOS app RectLabel.

sam2_polygon.mp4

Install Segment Anything Model 2 and download checkpoints.

Copy yaml files in sam2/configs/sam2.1 to sam2.

Put export_onnx.py and david-tomaseti-Vw2HZQ1FGjU-unsplash.jpg to the root folder.

To export an ONNX model.

python export_onnx.py --mode export

To check how the ONNX model works.

python export_onnx.py --mode import

You can download exported SAM 2.1 ONNX models.

• SAM 2.1 Tiny
• SAM 2.1 Small
• SAM 2.1 BasePlus
• SAM 2.1 Large

Download an ONNX Runtime folder.

• onnxruntime-osx-universal2-1.20.0.tgz for macOS
• onnxruntime-linux-x64-1.20.0.tgz for Ubuntu CPU
• onnxruntime-linux-x64-gpu-1.20.0.tgz for Ubuntu GPU

For Ubuntu, install gflags and opencv through vcpkg.

git clone https://github.com/microsoft/vcpkg.git
./vcpkg/bootstrap-vcpkg.sh
./vcpkg/vcpkg install gflags
./vcpkg/vcpkg install opencv

For Ubuntu GPU, install cuda and cudnn.

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install cuda-drivers
reboot
nvidia-smi

sudo apt install cuda-toolkit-11-8
vi ~/.bashrc
export PATH="/usr/local/cuda/bin${PATH:+:${PATH}}"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}"
source ~/.bashrc
which nvcc
nvcc --version

apt list libcudnn8 -a
cudnn_version=8.9.7.29
cuda_version=cuda11.8
sudo apt install libcudnn8=${cudnn_version}-1+${cuda_version}
sudo apt install libcudnn8-dev=${cudnn_version}-1+${cuda_version}
sudo apt install libcudnn8-samples=${cudnn_version}-1+${cuda_version}

Build and run.

# macOS
cmake -S . -B build -DONNXRUNTIME_ROOT_DIR=/Users/ryo/Downloads/onnxruntime-osx-universal2-1.20.0
# Ubuntu CPU
cmake -S . -B build -DONNXRUNTIME_ROOT_DIR=/root/onnxruntime-linux-x64-1.20.0 -DCMAKE_TOOLCHAIN_FILE=/root/vcpkg/scripts/buildsystems/vcpkg.cmake
# Ubuntu GPU
cmake -S . -B build -DONNXRUNTIME_ROOT_DIR=/root/onnxruntime-linux-x64-gpu-1.20.0 -DCMAKE_TOOLCHAIN_FILE=/root/vcpkg/scripts/buildsystems/vcpkg.cmake

cmake --build build

# macOS and Ubuntu CPU
./build/sam_cpp_test -encoder="sam2.1_tiny/sam2.1_tiny_preprocess.onnx" -decoder="sam2.1_tiny/sam2.1_tiny.onnx" -image="david-tomaseti-Vw2HZQ1FGjU-unsplash.jpg" -device="cpu"
# Ubuntu GPU
./build/sam_cpp_test -encoder="sam2.1_tiny/sam2.1_tiny_preprocess.onnx" -decoder="sam2.1_tiny/sam2.1_tiny.onnx" -image="david-tomaseti-Vw2HZQ1FGjU-unsplash.jpg" -device="cuda:0"