Learning to See Through Obstructions

My camera can see through fences: A deep learning approach for image de-fencing

In recent times, the availability of inexpensive image capturing devices such as smartphones/tablets has led to an exponential increase in the number of images/videos captured. However, sometimes the amateur photographer is hindered by fences in the scene which have to be removed after the image has been captured. Conventional approaches to image de-fencing suffer from inaccurate and non-robust fence detection apart from being limited to processing images of only static occluded scenes. In this paper, we propose a semi-automated de-fencing algorithm using a video of the dynamic scene. We use c…

Learning to See Through Obstructions with Layered Decomposition

We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera. Our method leverages motion differences between the background and obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. This learning-based layer reconstruction module facilitates accommodating potential errors in th…

Efficient Flow-Guided Multi-frame De-fencing

Taking photographs ''in-the-wild'' is often hindered by fence obstructions that stand between the camera user and the scene of interest, and which are hard or impossible to avoid. De-fencing is the algorithmic process of automatically removing such obstructions from images, revealing the invisible parts of the scene. While this problem can be formulated as a combination of fence segmentation and image inpainting, this often leads to implausible hallucinations of the occluded regions. Existing multi-frame approaches rely on propagating information to a selected keyframe from its temporal neighb…

学习型分频光谱金属镜用于可见光波段无遮挡广谱成像

De-occluding broadband metalens

本研究提出学习型分频光谱金属镜，实现可见光波段无遮挡广谱成像，提升物体检测和分割精度。

用于突发图像融合与图层分离的神经样条场

Neural Spline Fields for Burst Image Fusion and Layer Separation

提出基于神经样条场的双层alpha合成表示，在测试时优化中联合融合突发图像并分解为透射层与遮挡层，无需后处理即可超越现有遮挡去除方法。