具有上下文感知的局部高斯泼溅编辑
Localized Gaussian Splatting Editing with Contextual Awareness
AI总结：
本文提出一种光照感知的三维高斯泼溅场景编辑管线，通过锚点视图提议和深度引导修复得分蒸馏采样实现局部物体编辑，在无需显式光传输建模的情况下保持全局光照一致性。
摘要：
近期，利用扩散先验进行文本引导的单个三维物体生成已取得巨大成功。然而，这些方法并不适用于物体插入和替换任务，因为它们没有考虑背景，导致环境中的光照不匹配。为了弥补这一差距，我们提出了一种面向三维高斯泼溅（3DGS）表示的光照感知三维场景编辑管线。我们的关键观察是，最先进的条件二维扩散模型所进行的图像修复在光照方面与背景保持一致。为了利用训练良好的扩散模型的先验知识进行三维物体生成，我们的方法采用了一种从粗到细的物体优化管线，并使用修复后的视图。在第一个粗略步骤中，给定一个理想的修复视图，我们实现了图像到三维的升维。该过程利用了来自视图条件扩散模型的三维感知扩散先验，该先验保留了条件图像中存在的光照。为了获得理想的修复图像，我们引入了锚点视图提议（AVP）算法，以找到最能代表目标区域场景光照的单个视图。在第二个纹理增强步骤中，我们引入了一种新颖的深度引导修复得分蒸馏采样（DI-SDS），它利用修复扩散先验增强了几何和纹理细节，超越了第一个粗略步骤中三维感知扩散先验知识的范围。DI-SDS不仅提供了细粒度的纹理增强，还促使优化过程尊重场景光照。我们的方法无需显式建模光传输，即可高效实现具有全局光照一致性的局部编辑。我们通过评估包含明显高光和阴影的真实场景中的编辑效果，证明了我们方法的鲁棒性，并与最先进的文本到三维编辑方法进行了比较。
英文摘要：
Recent text-guided generation of individual 3D object has achieved great success using diffusion priors. However, these methods are not suitable for object insertion and replacement tasks as they do not consider the background, leading to illumination mismatches within the environment. To bridge the gap, we introduce an illumination-aware 3D scene editing pipeline for 3D Gaussian Splatting (3DGS) representation. Our key observation is that inpainting by the state-of-the-art conditional 2D diffusion model is consistent with background in lighting. To leverage the prior knowledge from the well-trained diffusion models for 3D object generation, our approach employs a coarse-to-fine objection optimization pipeline with inpainted views. In the first coarse step, we achieve image-to-3D lifting given an ideal inpainted view. The process employs 3D-aware diffusion prior from a view-conditioned diffusion model, which preserves illumination present in the conditioning image. To acquire an ideal inpainted image, we introduce an Anchor View Proposal (AVP) algorithm to find a single view that best represents the scene illumination in target region. In the second Texture Enhancement step, we introduce a novel Depth-guided Inpainting Score Distillation Sampling (DI-SDS), which enhances geometry and texture details with the inpainting diffusion prior, beyond the scope of the 3D-aware diffusion prior knowledge in the first coarse step. DI-SDS not only provides fine-grained texture enhancement, but also urges optimization to respect scene lighting. Our approach efficiently achieves local editing with global illumination consistency without explicitly modeling light transport. We demonstrate robustness of our method by evaluating editing in real scenes containing explicit highlight and shadows, and compare against the state-of-the-art text-to-3D editing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
