My camera can see through fences: A deep learning approach for image de-fencing

VirtualFencer：基于从自然视频中提取的策略生成击剑回合

VirtualFencer: Generating Fencing Bouts based on Strategies Extracted from In-the-Wild Videos

VirtualFencer从自然视频中无监督提取3D击剑动作与策略，并据此生成逼真的击剑行为，支持自我博弈、与真实视频动作对弈及与职业选手交互对弈。

Video De-fencing

This paper describes and provides an initial solution to a novel video editing task, i.e., video de-fencing. It targets automatic restoration of the video clips that are corrupted by fence-like occlusions during capture. Our key observation lies in the visual parallax between fences and background scenes, which is caused by the fact that the former are typically closer to the camera. Unlike in traditional image inpainting, fence-occluded pixels in the videos tend to appear later in the temporal dimension and are therefore recoverable via optimized pixel selection from relevant frames. To event…

Accurate and efficient video de-fencing using convolutional neural networks and temporal information

De-fencing is to eliminate the captured fence on an image or a video, providing a clear view of the scene. It has been applied for many purposes including assisting photographers and improving the performance of computer vision algorithms such as object detection and recognition. However, the state-of-the-art de-fencing methods have limited performance caused by the difficulty of fence segmentation and also suffer from the motion of the camera or objects. To overcome these problems, we propose a novel method consisting of segmentation using convolutional neural networks and a fast/robust recov…

Efficient Flow-Guided Multi-frame De-fencing

Taking photographs ''in-the-wild'' is often hindered by fence obstructions that stand between the camera user and the scene of interest, and which are hard or impossible to avoid. De-fencing is the algorithmic process of automatically removing such obstructions from images, revealing the invisible parts of the scene. While this problem can be formulated as a combination of fence segmentation and image inpainting, this often leads to implausible hallucinations of the occluded regions. Existing multi-frame approaches rely on propagating information to a selected keyframe from its temporal neighb…

Creating awareness about security and safety on highways to mitigate wildlife-vehicle collisions by detecting and recognizing wildlife fences using deep learning and drone technology

In South Africa, it is a common practice for people to leave their vehicles beside the road when traveling long distances for a short comfort break. This practice might increase human encounters with wildlife, threatening their security and safety. Here we intend to create awareness about wildlife fencing, using drone technology and computer vision algorithms to recognize and detect wildlife fences and associated features. We collected data at Amakhala and Lalibela private game reserves in the Eastern Cape, South Africa. We used wildlife electric fence data containing single and double fences …

深度生成对抗网络用于单图像遮挡移除

Deep Generative Adversarial Network for Occlusion Removal from a Single Image

本研究提出一种全自动两阶段卷积神经网络，利用生成对抗网络从单张图像中检测并移除遮挡（如栅栏），在自建数据集上验证了零样本泛化能力。

Learning to See Through Obstructions

We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions or raindrops, from a short sequence of images captured by a moving camera. Our method leverages the motion differences between the background and the obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. The learning-based layer reconstruction allows us to accommodate potential errors in the flow esti…

Learning to See Through Obstructions with Layered Decomposition

We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera. Our method leverages motion differences between the background and obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. This learning-based layer reconstruction module facilitates accommodating potential errors in th…

Semantic Segmentation for Urban-Scene Images

Urban-scene Image segmentation is an important and trending topic in computer vision with wide use cases like autonomous driving [1]. Starting with the breakthrough work of Long et al. [2] that introduces Fully Convolutional Networks (FCNs), the development of novel architectures and practical uses of neural networks in semantic segmentation has been expedited in the recent 5 years. Aside from seeking solutions in general model design for information shrinkage due to pooling, urban-scene image itself has intrinsic features like positional patterns [3]. Our project seeks an advanced and integra…

深度学习模型在视觉识别任务中对部分物体遮挡是否具有鲁棒性？

Are Deep Learning Models Robust to Partial Object Occlusion in Visual Recognition Tasks?

该研究构建了含真实与人工遮挡的IRUO数据集，结合人类认知实验对比了CNN、ViT等模型在不同遮挡下的识别鲁棒性，发现ViT表现更优但漫射遮挡对CNN类模型影响显著。