SageRef

SageRef: Single Image Reflection Removal

• SageRef
• Usage
• Datasets Used
• Evaluation
  • Predictions
  • Performance Metrics
  • Models used for comparisons

Usage

$ python main.py --help
usage: main.py [-h] --gpu GPU --mode {train,eval,predict} [--epochs EPOCHS] [--latent_dim LATENT_DIM] [--out_dir OUT_DIR]
               [--data_dir DATA_DIR] [--split_dir SPLIT_DIR] [--batch_size BATCH_SIZE] [--learning_rate LEARNING_RATE]

run the relection removal experiment

optional arguments:
  -h, --help            show this help message and exit
  --gpu GPU             gpu id
  --mode {train,eval,predict}
                        mode: [train, eval, predict]
  --epochs EPOCHS       number of training epochs
  --latent_dim LATENT_DIM
                        latent space feature dimensions
  --out_dir OUT_DIR     output directory
  --data_dir DATA_DIR   data directory
  --split_dir SPLIT_DIR
                        data directory
  --batch_size BATCH_SIZE
                        batch size for training
  --learning_rate LEARNING_RATE
                        learning rate

Datasets Used

• SIR2 Benchmark Dataset
• CeilNet
• Dataset by Zhang et al

Evaluation

Predictions

# run the following code after defining img_path and module
from src.utils import predict
img = Image.open(img_path)
predict(module, img)

Performance Metrics

• PSNR (Peak Signal-to-Noise Ratio): torchmetrics.PeakSignalNoiseRatio
• SSIM (Structural Similarity Index): torchmetrics.StructuralSimilarityIndexMeasure
• LPIPS (Learned Perceptual Image Patch Similarity): torchmetrics.image.lpip.LearnedPerceptualImagePatchSimilarity

Models used for comparisons

• "Robust Reflection Removal with Reflection-free Flash-only Cues" (CVPR 2021) (pdf, github)
• "Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation" (NeurIPS 2021) (pdf, github)
• "Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements" (CVPR 2019) (pdf,github)
• "Single Image Reflection Removal with Perceptual Losses" (CVPR 2018) (pdf, github)

Acknowledgement

The network architecture and the codebase is based on Phillip Lippe's UvA Deep Learning Tutorial 9: Deep Autoencoders