Identity Enhanced Residual Image Denoising

This repository is for Identity Enhanced Residual Image Denoising (IERD) introduced in the following paper

Saeed Anwar, Cong Phuoc Huynh, Fatih Porikli, "Identity Enhanced Residual Image Denoising", IEEE Computer Vision and Pattern Recognition Workshop, CVPRw, 2020

Contents

1. Introduction
2. Network
3. Requirements
4. Test
5. Results
6. Citation
7. Acknowledgements

Introduction

We propose to learn a fully-convolutional network model that consists of a Chain of Identity Mapping Modules and residual on the residual architecture for image denoising. Our network structure possesses three distinctive features that are important for the noise removal task. Firstly, each unit employs identity mappings as the skip connections and receives pre-activated input to preserve the gradient magnitude propagated in both the forward and backward directions. Secondly, by utilizing dilated kernels for the convolution layers in the residual branch, each neuron in the last convolution layer of each module can observe the full receptive field of the first layer. Lastly, we employ the residual on the residual architecture to ease the propagation of the high-level information. Contrary to current state-of-the-art real denoising networks, we also present a straightforward and single-stage network for real image denoising.

The proposed network produces remarkably higher numerical accuracy and better visual image quality than the classical state-of-the-art and CNN algorithms when being evaluated on the three conventional benchmark and three real-world datasets

Denoising results: In the first row, an image is corrupted by the Gaussian noise with σ = 50 from the BSD68 dataset. In the second row, a sample image from the RNI15 real noisy dataset. Our results have the best PSNR score for synthetic images, and unlike other methods, it does not have over-smoothing or over-contrasting artifacts.

Network

The proposed network architecture, which consists of multiple modules with similar structures. Each module is composed of a series of pre-activation-convolution layer pairs. The multiplier block negates the input block features to be summed at the end of the mapping module

Requirements

• The model is built in PyTorch 0.4.0, PyTorch 0.4.1
• Tested on Ubuntu 14.04/16.04 environment
• python 3.6
• CUDA 9.0
• cuDNN 5.1
• pytorch=0.4.1
• torchvision=0.2.1
• imageio
• pillow
• matplotlib
• tqdm

Test

Quick start

1. Download the trained models and code of our paper from Google Drive. The total size for all models is 3.1MB.

2. cd to '/IERDTestCode/code', run the following scripts and find the results in directory IERD_Results.

   You can use the following script to test the algorithm. The first script is without self-ensembler and the second one is with self-ensemble.

CUDA_VISIBLE_DEVICES=0 python main.py --data_test MyImage --noise_g 1 --model IERD --n_feats 64 --pre_train ../trained_model/IERD.pt --test_only --save_results --save 'SSID_Results' --testpath ../noisy --testset SIDD

CUDA_VISIBLE_DEVICES=0 python main.py --data_test MyImage --noise_g 1 --model IERD --n_feats 64 --pre_train ../trained_model/IERD.pt --test_only --save_results --save 'SSIDPlus_Results' --testpath ../noisy --testset SIDD --self_ensemble

Results

All the results for IERD can be downloaded from GoogleDrive from SSID (118MB), RNI15 (9MB) and DnD (2.3GB).

DnD Results

Comparison of our method against the state-of-the-art algorithms on real images containing Gaussian noise from Darmstadt Noise Dataset (DND) benchmark for different denoising algorithms. Difference can be better viewed in magnified view.

Mean PSNR and SSIM of the denoising methods evaluated on the real images dataset

SSID Results

A few challenging examples from SSID dataset. Our method can restore true colors and remove noise.

The quantitative results (in PSNR (dB)) for the SSID dataset

RNI15

Sample visual examples from RNI15. Our method annihilates the noise and preserves the essential details while the competing methods fail to deliver satisfactory results i.e. unable to remove noise. Best viewed on high-resolution display.

Ablation Studies

Effect of Patch Size

Layers vs. Dilation

Effect of Number of Modules

Effect of Each Component

Structure of Identity Module

For more information, please refer to our paper

Citation

If you find the code helpful in your resarch or work, please cite the following papers.

@inproceedings{anwar2020IERD,
  title={Identity Enhanced Residual Image Denoising},
  author={Anwar, Saeed and Phuoc Huynh, Cong and Porikli, Fatih},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  pages={520--521},
  year={2020}
}

@article{anwar2019ridnet,
  title={Real Image Denoising with Feature Attention},
  author={Anwar, Saeed and Barnes, Nick},
  journal={IEEE International Conference on Computer Vision (ICCV-Oral)},
  year={2019}
}

Acknowledgements

This code is built on DRLN (PyTorch)