GSFix3D：基于扩散引导的高斯溅射新视角修复
GSFix3D: Diffusion-Guided Repair of Novel Views in Gaussian Splatting
AI总结：
针对3D高斯溅射在极端视角或部分观测区域生成高质量新视角渲染的挑战，以及扩散模型依赖文本提示和缺乏场景信息的局限，提出GSFix3D框架，通过将扩散模型先验知识蒸馏到3D表示中，提升欠约束区域视觉保真度并保留场景一致性，实验验证其最先进性能。
摘要：
近期3D高斯溅射技术的发展显著提升了新视角合成效果，但从极端新视角或部分观测区域生成高质量渲染仍具挑战。同时，扩散模型虽有强大生成能力，却依赖文本提示且缺乏特定场景信息感知，阻碍了精准3D重建任务。为解决这些局限，我们提出GSFix3D框架，通过将扩散模型的先验知识蒸馏到3D表示中，在保留观测场景细节一致性的同时，提升欠约束区域的视觉保真度。其核心是GSFixer，一种通过定制微调协议获得的潜在扩散模型，可利用网格和3D高斯将预训练生成模型适配到不同重建方法的多种环境和伪影类型，实现对未见过相机姿态的稳健新视角修复。此外，我们提出随机掩码增强策略，使GSFixer能合理修复缺失区域。在挑战性基准上的实验表明，GSFix3D和GSFixer达到了最先进性能，仅需对捕获数据进行最小化场景特定微调。真实世界测试进一步证实其对潜在姿态误差的鲁棒性。我们的代码和数据将公开。
英文摘要：
Recent developments in 3D Gaussian Splatting have significantly enhanced novel view synthesis, yet generating high-quality renderings from extreme novel viewpoints or partially observed regions remains challenging. Meanwhile, diffusion models exhibit strong generative capabilities, but their reliance on text prompts and lack of awareness of specific scene information hinder accurate 3D reconstruction tasks. To address these limitations, we introduce GSFix3D, a novel framework that improves the visual fidelity in under-constrained regions by distilling prior knowledge from diffusion models into 3D representations, while preserving consistency with observed scene details. At its core is GSFixer, a latent diffusion model obtained via our customized fine-tuning protocol that can leverage both mesh and 3D Gaussians to adapt pretrained generative models to a variety of environments and artifact types from different reconstruction methods, enabling robust novel view repair for unseen camera poses. Moreover, we propose a random mask augmentation strategy that empowers GSFixer to plausibly inpaint missing regions. Experiments on challenging benchmarks demonstrate that our GSFix3D and GSFixer achieve state-of-the-art performance, requiring only minimal scene-specific fine-tuning on captured data. Real-world test further confirms its resilience to potential pose errors. Our code and data will be made publicly available. Project page: https://gsfix3d.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
