学习合成逼真雨滴用于单图像雨滴去除
Learning from synthetic photorealistic raindrop for single image raindrop removal
浏览论文内容
中文总结 AI 辅助
针对雨天图像中雨滴干扰问题，提出首个基于物理渲染的合成逼真雨滴数据集，并设计感知折射与模糊的检测网络及恢复结构的去除网络，实现单图像雨滴去除的最先进性能。
中文摘要 AI 辅助
在雨天场景中，附着在相机镜头或挡风玻璃上的雨滴是不可避免的，并且可能对许多计算机视觉系统（如自动驾驶）造成问题。由于雨滴的外观受太多参数影响，因此不太可能找到基于模型的有效解决方案。基于学习的方法也存在问题，因为传统的学习方法无法正确建模复杂的外观。而深度学习方法缺乏足够大且逼真的训练数据。为了解决这个问题，在我们的工作中，我们提出了第一个用于训练的合成附着雨滴的照片级真实感数据集。渲染是基于物理的，考虑了水的动力学、几何和光度。数据集包含各种类型的雨天场景，特别是雨天驾驶场景。基于雨滴图像的建模，我们引入了一个检测网络，该网络具有对雨滴折射及其模糊的感知能力。在此基础上，我们提出了能够很好恢复图像结构的去除网络。严格的实验证明了我们提出的框架的先进性能。
英文摘要
Raindrops adhered to camera lens or windshield are inevitable in rainy scenes and can become an issue for many computer vision systems such as autonomous driving. Because raindrop appearance is affected by too many parameters, therefore it is unlikely to find an effective model based solution. Learning based methods are also problematic, because traditional learning method cannot properly model the complex appearance. Whereas deep learning method lacks sufficiently large and realistic training data. To solve it, in our work, we propose the first photo-realistic dataset of synthetic adherent raindrops for training. The rendering is physics based with consideration of the water dynamic, geometric and photometry. The dataset contains various types of rainy scenes and particularly the rainy driving scenes. Based on the modeling of raindrop imagery, we introduce a detection network which has the awareness of the raindrop refraction as well as its blurring. Based on that, we propose the removal network that can well recover the image structure. Rigorous experiments demonstrate the state-of-the-art performance of our proposed framework.
发表机构
- State Key Laboratory of VR Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）
- Data61-CSIRO（澳大利亚联邦科学与工业研究组织Data61）
- Tencent（腾讯）
- Australian National University（澳大利亚国立大学）
- Peng Cheng Laboratory（鹏城实验室）
机构由 AI 辅助整理，请以论文原文为准。
