从合成到去除：基于物理的反射模拟与基于扩散的视频去反射
From Synthesis to Removal: Physics-Grounded Reflection Simulation and Diffusion-Based Video Dereflection
浏览论文内容
中文总结 AI 辅助
该研究针对视频反射去除数据与模型缺失问题，提出闭环框架S2R，含基于物理的反射模拟、首个扩散式视频去反射模型及专用基准，实现了更优性能与更快推理。
中文摘要 AI 辅助
通过玻璃拍摄的视频常包含反射，这会降低视觉质量并干扰下游视觉任务。尽管单图像反射去除已被广泛研究，但视频反射去除仍未得到充分探索，原因在于缺乏配对视频数据、时间一致的去除模型以及专用评估基准。我们提出了一个闭环框架，将基于物理的反射模拟、基于扩散的视频去反射和基准评估统一起来。我们的S2R-Synthesis流水线通过在结构空间中执行基于物理的增强，并使用训练好的视频扩散渲染器渲染真实的反射视频，生成配对的反射视频和无反射视频；该增强涵盖了与玻璃相关的关键效应，包括粗糙度诱导的模糊、厚度诱导的重影以及反射率变化。基于合成数据，我们推出了S2R-Removal，这是首个基于扩散的视频反射去除模型，它通过感知反射的潜在空间适配和单步像素-几何细化来适配预训练的视频扩散先验，在单次去噪步骤中恢复干净的透射图像。我们进一步构建了S2R-Bench，这是首个视频反射去除基准，支持全参考评估和真实世界人类感知评估。在S2R-Bench和多个公开图像基准上的实验表明，该模型达到了最先进的性能，推理速度甚至比非扩散基线更快，且验证了S2R-Synthesis的有效性。项目页面：this https URL。
英文摘要
Videos captured through glass often contain reflections that degrade visual quality and interfere with downstream vision tasks. Although single-image reflection removal has been extensively studied, video reflection removal remains largely underexplored due to the lack of paired video data, temporally coherent removal models, and dedicated evaluation benchmarks. We present a closed-loop framework that unifies physics-grounded reflection simulation, diffusion-based video dereflection, and benchmark evaluation. Our S2R-Synthesis pipeline generates paired reflected and reflection-free videos by performing physics-grounded augmentation in the structure space and rendering realistic reflected videos with a trained video diffusion renderer; the augmentation models key glass-related effects including roughness-induced blur, thickness-induced ghosting, and reflectance variation. Based on the synthesized data, we introduce S2R-Removal, the first diffusion-based video reflection removal model, which adapts a pretrained video diffusion prior through reflection-aware latent adaptation and one-step pixel-geometric refinement, recovering the clean transmission in a single denoising step. We further build S2R-Bench, the first benchmark for video reflection removal, supporting both full-reference evaluation and real-world human perceptual assessment. Experiments on S2R-Bench and multiple public image benchmarks demonstrate state-of-the-art performance and faster inference than even non-diffusion baselines, and validate the effectiveness of S2R-Synthesis. Project page: https://codingwzp.github.io/VideoDereflection_S2R.
发表机构
- Xiaomi Inc.（小米公司）
- MiLM Plus
机构由 AI 辅助整理，请以论文原文为准。
