发表机构
Huanjiang Laboratory; Zhejiang University; Suzhou University of Science and Technology（浣江实验室; 浙江大学; 苏州科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出LLPR框架，通过嵌入位置感知学习分支（推理时可移除）和基于物理模型的重建方案，在不增加推理成本的情况下提升单图像雨滴去除性能，并在真实数据集上取得优于现有方法的结果。
AI 中文摘要
雨滴附着在窗户或相机镜头上，会导致背景场景出现遮挡和失真。现有的雨滴去除方法主要集中于设计复杂的CNN或Transformer架构来恢复失真和缺失的纹理。在本文中，我们尝试将位置信息和物理模型集成到现成的CNN或Transformer架构中，以帮助提升其性能。具体而言，我们注意到现有方法部署了一个预处理子网络来生成二进制或软掩码以指示雨滴位置，这会增加网络参数和计算复杂度。相比之下，我们嵌入了一个位置感知学习分支，在训练阶段教导编码器具备感知雨滴位置的能力。需要注意的是，该位置感知学习分支在推理过程中可以被移除（在无额外成本的情况下实现性能提升）。此外，我们不是直接重建无雨滴图像（即背景场景），而是设计了一种基于物理的重建方案，首先学习透明度矩阵和雨滴层。然后，基于物理模型反向推导出潜在背景层。通过结合上述组件，我们提出了位置感知学习与物理重建（LLPR）框架来解决这一具有挑战性的不适定问题。我们还收集了一个真实世界的雨滴退化图像数据集，这对单图像雨滴去除（SIRR）方法具有挑战性。大量实验结果表明，我们的LLPR框架的有效性和通用性，在性能上优于最先进的SIRR方法。代码将在论文被接收后公开。
英文摘要
Raindrops can cause occlusion and distortion in the background scenes due to their adherence to windows or camera lenses. Existing raindrop removal methods concentrate on designing sophisticated CNN or Transformer architectures to recover distorted and missing texture. In this paper, we try to integrate location information and physical model into off-the-shelf CNN or Transformer architectures to help improve their performance. Specifically, we notice that existing methods deploy a preprocessing sub-network to generate a binary or soft mask to indicate the raindrop location, which will increase the network parameters and computational complexity. In contrast, a location-aware learning branch is embedded to teach the encoder in the training phase with the capability of perceiving the position of the raindrops. Note that this location-aware learning branch can be removed during the inference process (achieving performance improvements at no cost). Furthermore, instead of directly reconstructing the raindrop-free image (i.e., background scene), we devise a physics-based reconstruction scheme to first learn the transparency matrix and the raindrop layer. The latent background layer is then reversely derived based on the physical model. By combining the above-mentioned components, we propose our location-aware learning and physics-based reconstruction (LLPR) framework for this challenging ill-posed problem. We also collect a real-world raindrop-degraded image dataset, which is challenging for single-image raindrop removal (SIRR) methods. Extensive experimental results demonstrate the effectiveness and generality of our LLPR framework, achieving superior performance against state-of-the-art SIRR methods. The code will be made available upon acceptance.
