ViFu：通过可见部分融合实现具有干净背景的多个360°物体重建
ViFu: Multiple 360$^\circ$ Objects Reconstruction with Clean Background via Visible Part Fusion
AI总结：
针对多时间戳场景中遮挡导致背景与360°物体重建不完整的问题，提出ViFu，通过分割对齐和可见性感知的辐射场融合恢复干净背景与完整物体。
摘要：
本文提出一种方法，用于从不同时间戳的场景观测中分割并恢复静态、干净的背景以及多个360°物体。近期工作已使用神经辐射场对3D场景建模，并提升了新视角合成的质量，但很少有研究关注对训练图像中不可见或被遮挡部分的建模。这些重建不足的部分同时限制了场景编辑和渲染视角选择，从而限制了其在为下游任务生成合成数据方面的效用。我们的基本思想是，通过观察同一组物体在不同排列下的状态，使一个场景中不可见的部分可能在其他场景中变为可见。通过融合来自每个场景的可见部分，可以实现背景和前景物体的无遮挡渲染。我们将多场景融合任务分解为两个主要组成部分：（1）物体/背景分割与对齐，在此我们利用针对我们新颖问题形式定制的基于点云的方法；（2）辐射场融合，在此我们引入可见性场以量化辐射场的可见信息，并提出可见性感知渲染用于一系列场景的融合，最终获得干净背景和360°物体渲染。在合成数据集和真实数据集上进行了全面实验，结果证明了我们方法的有效性。
英文摘要：
In this paper, we propose a method to segment and recover a static, clean background and multiple 360$^\circ$ objects from observations of scenes at different timestamps. Recent works have used neural radiance fields to model 3D scenes and improved the quality of novel view synthesis, while few studies have focused on modeling the invisible or occluded parts of the training images. These under-reconstruction parts constrain both scene editing and rendering view selection, thereby limiting their utility for synthetic data generation for downstream tasks. Our basic idea is that, by observing the same set of objects in various arrangement, so that parts that are invisible in one scene may become visible in others. By fusing the visible parts from each scene, occlusion-free rendering of both background and foreground objects can be achieved. We decompose the multi-scene fusion task into two main components: (1) objects/background segmentation and alignment, where we leverage point cloud-based methods tailored to our novel problem formulation; (2) radiance fields fusion, where we introduce visibility field to quantify the visible information of radiance fields, and propose visibility-aware rendering for the fusion of series of scenes, ultimately obtaining clean background and 360$^\circ$ object rendering. Comprehensive experiments were conducted on synthetic and real datasets, and the results demonstrate the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
