镜头光晕去除与重建
Lens Flare Removal and Reconstruction
查看机构详情
- Meta Reality Labs Research（Meta现实实验室）
- Meta Reality Labs（Meta现实实验室）
- TU Munich, MCML（慕尼黑工业大学，慕尼黑机器学习中心）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
针对大型镜头光晕去除与多视角重建问题，构建新数据集并微调扩散模型去除光晕，同时提出利用对称性的光晕表示模型与3DGS联合优化，实现场景分解及光晕可编辑迁移。
中文摘要 AI 辅助
图像中镜头光晕的存在会显著降低下游应用（如三维场景重建）的结果质量，这是因为镜头光晕是相机成像系统的属性，而非所建模场景的一部分。已有方法致力于去除光源周围的小型光晕，但现有方法在处理大型光晕（例如充满整个图像的光晕）时表现不佳。在本工作中，我们构建了一个用于大型光晕去除的新数据集，结合了公开的真实世界数据与程序化生成流程。我们在该数据集上微调了一个基于扩散的模型，以去除复杂的大型镜头光晕。另一方面，镜头光晕仍是有效的艺术工具，在媒体中被广泛使用。虽然有方法可以模拟二维光晕，但跨多个视角一致地表示和重建镜头光晕尚未被探索。为实现这一目标，我们引入了一个利用镜头光晕关于相机主点对称性的光晕表示模型。我们提出了一种计算流程，用于联合优化该光晕模型和高斯溅射模型（3DGS）。这使我们能够利用光晕去除模型，将三维场景分解为镜头光晕和场景本身。由于重建的光晕是显式且可重新渲染的，它可以被编辑并迁移到新图像和新的三维场景中。我们在一个既有基准和一个用于大型反射性光晕的新基准上评估了去除效果，直接量化了光晕/场景分解，并展示了该流程对自动光源定位误差的鲁棒性。
英文摘要
The presence of lens flares in images can significantly reduce the quality of downstream application results for tasks such as 3D scene reconstruction. This is because lens flares are a property of the camera imaging system, and not a part of the underlying scene being modeled. There are previous methods that tackle the removal of small flares focused around a light source. However, existing methods struggle with large flares, such as those that fill the entire image. In this work, we compile a novel dataset for large-flare removal, combining publicly available real-world data with a procedural generation pipeline. We fine-tune a diffusion-based model on our dataset to remove complex, large lens flares. On the other hand, lens flares remain effective artistic tools, widely used in the media. While there are ways to simulate 2D flares, representing and reconstructing lens flares consistently across multiple views has not yet been explored. To achieve this, we introduce a flare representation model that leverages the symmetry of lens flares about the camera's principal point. We propose a computational pipeline to jointly optimize this flare model and a Gaussian splatting model (3DGS). This enables the decomposition of a 3D scene into lens flares and the scene itself, using our flare-removal model. Because the reconstructed flare is explicit and re-renderable, it can be edited and transferred to novel images and new 3D scenes. We evaluate removal on an established benchmark and a new one for large reflective flares, quantify the flare/scene decomposition directly, and show that the pipeline is robust to errors in automatic light-source localization.
