Polarized Reflection Removal with Perfect Alignment in the Wild
CVPR 2020

Abstract

We present a novel formulation to removing reflection from polarized images in the wild. We first identify the misalignment issues of existing reflection removal datasets where the collected reflection-free images are not perfectly aligned with input mixed images due to glass refraction. Then we build a new dataset with more than 100 types of glass in which obtained transmission images are perfectly aligned with input mixed images. Second, capitalizing on the special relationship between reflection and polarized light, we propose a polarized reflection removal model with a two-stage architecture. In addition, we design a novel perceptual NCC loss that can improve the performance of reflection removal and general image decomposition tasks. We conduct extensive experiments, and results suggest that our model outperforms state-of-the-art methods on reflection removal.

Technical Video

Motivation

  • A common issue of many existing reflection removal methods [34, 32, 7, 31, 24] is that strict assumptions are imposed on reflection.
  • The lack of diverse and high quality real-world data is another challenging issue.

Key design

  • Polarization information

    • To be able to relax the assumptions about the appearance of reflection, we leverage polarization that inherently exists in almost all reflected light

  • A method to collect perfect-alignment real-world GT for reflection removal

    • We design a new data collection pipeline called M-R, which helps us collect diverse real-world data with perfect alignment by utilizing glass in the real world.

  • Two-stage framework for reflection removal

    • We propose a two-stage framework for reflection removal from polarized images. Our approach firstly estimates reflection, with which it infers the transmission image secondly. This framework helps us improve 2dB PSNR on our real-world dataset.

    BibTeX