ORCA：面向新视角合成的遮挡感知细化与补全
ORCA: Occlusion-Aware Refinement and Completion for Novel View Synthesis
- Wrocław University of Science and Technology（弗罗茨瓦夫理工大学）
- Jagiellonian University（雅盖隆大学）
- IDEAS Research Institute（IDEAS研究所）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
ORCA提出一种遮挡感知的新视角合成方法，利用单目深度和RGB-D信息修复小间隙，仅在必要时使用生成式修复，减少幻觉并提升重建质量。
中文摘要 AI 辅助
从单张图像进行新视角合成是一个根本性的模糊问题。当相机远离输入视角移动时，先前隐藏的区域变得可见，从而暴露出重建场景中缺失的几何结构和空洞。现有方法通常依赖生成模型来补全这些区域。然而，许多此类伪影是深度边界附近的小间隙，并不需要生成新的场景内容。为了消除生成图像的昂贵过程，我们引入了ORCA，一种从单张图像重建和补全可探索3D场景的遮挡感知方法。ORCA首先利用单目深度将3D结构引入高斯锚点表示中，同时保留原始相机射线对应关系。在场景探索过程中，缺失区域根据其大小和结构进行处理。小的去遮挡区域使用重建中已有的RGB-D信息进行修复，而生成式修复则保留给无法从场景中可靠恢复的较大区域。新增的高斯锚点被添加并在本地优化，而不修改现有表示。通过减少对生成式修复的不必要依赖，ORCA限制了生成引起的幻觉，并更好地保留了原始场景的内容和结构。在DIV2K上，ORCA在所有报告的指标上均优于VistaDream，将MUSIQ从61.60提高到68.71，CLIP-IQA从0.474提高到0.574。这些结果表明，许多新视角伪影可以通过重用重建场景中已有的信息来有效修复。
英文摘要
Novel-view synthesis from a single image is a fundamentally ambiguous problem. As the camera moves away from the input viewpoint, previously hidden regions become visible, exposing missing geometry and holes in the reconstructed scene. Existing methods often rely on generative models to complete such regions. However, many of these artifacts are small gaps near depth boundaries and do not require generating new scene content. In order to eliminate expensive process of generating image we introduce ORCA, an occlusion-aware method for reconstructing and completing explorable 3D scenes from a single image. ORCA first introduces 3D structure into a Gaussian-anchor representation using monocular depth while preserving the original camera-ray correspondence. During scene exploration, missing regions are handled based on their size and structure. Small disocclusions are repaired using RGB-D information already available in the reconstruction, while generative inpainting is reserved for larger regions that cannot be reliably recovered from the scene. New Gaussian anchors are added and optimized locally without modifying the existing representation. By reducing unnecessary reliance on generative inpainting, ORCA limits generation-induced hallucinations and better preserves the content and structure of the original scene. On DIV2K, ORCA improves novel-view quality over VistaDream across all reported metrics, increasing MUSIQ from 61.60 to 68.71 and CLIP-IQA from 0.474 to 0.574. These results show that many novel-view artifacts can be repaired effectively by reusing information already present in the reconstructed scene.
