This repository contains the PyTorch code for our paper "LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network" by Hao Yang, Liyuan Pan, Yan Yang, Richard Hartley, and Miaomiao Liu.
Recovering sharp images from dual-pixel (DP) pairs with disparity-dependent blur is a challenging task. Existing blur map-based deblurring methods have demonstrated promising results. In this paper, we propose, to the best of our knowledge, the first framework that introduces the contrastive language-image pre-training framework (CLIP) to accurately estimate the blur map from a DP pair unsupervisedly. To achieve this, we first carefully design text prompts to enable CLIP to understand blur-related geometric prior knowledge from the DP pair. Then, we propose a format to input a stereo DP pair to CLIP without any fine-tuning, despite the fact that CLIP is pre-trained on monocular images. Given the estimated blur map, we introduce a blur-prior attention block, a blur-weighting loss, and a blur-aware loss to recover the all-in-focus image. Our method achieves state-of-the-art performance in extensive experiments (see Fig. 1).
Place the image in the input folder and run single_image.py for single image blur map estimation and dp_image.py for DP image blur map estimation.
@inproceedings{yang2024ldp,
title={LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network},
author={Yang, Hao and Pan, Liyuan and Yang, Yan and Hartley, Richard and Liu, Miaomiao},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={24078--24087},
year={2024}
}
The code is borrowed from the following repositories, thanks for sharing.