arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

ϕ-RIE:从照片级真实重建到交互式环境

ϕ-RIE: From Photorealistic Reconstruction to Interactive Environments

Runyi Yang, Deheng Zhang, Xiaoye Wang, Kanzhi Wu, Lei Sun, Ajad Chhatkuli, Kunyu Peng, Luc Van Gool, Danda Pani Paudel

arXiv 2609.26795首次发表:更新:

发表机构

INSAIT, Sofia University “St. Kliment Ohridski”; vivo Mobile Communication Co., Ltd.; Karlsruhe Institute of Technology(索非亚大学圣克利门特奥赫里德大学INSAIT研究所; 维沃移动通信有限公司; 卡尔斯鲁厄理工学院)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

针对3DGS重建缺乏物理交互支持的问题,提出ϕ-RIE流水线,通过耦合资产构建与源移除,将选定对象转为可移动资产并补全背景,在ScanNet++上提升匹配F1至0.383,实现交互式场景转换。

AI 中文摘要

3D高斯泼溅(3DGS)能够以照片级真实感重建所捕获的场景,但由此得到的表示本身并不支持物理交互。相反,机器人仿真需要对象级别的变化,即对象必须独立移动、发生接触,并揭示先前被遮挡的周围环境。这一差距的产生是因为对象外观可能仍与背景纠缠在一起,而隐藏的对象几何形状和被遮挡的背景内容可能未被观测到。为应对这一挑战,我们提出了ϕ-RIE,一种高斯原生的流水线,可将选定的对象转换为可移动的仿真器资产,同时保留其余的重建。我们的关键观察是,资产构建和源移除应当耦合,即一个对象身份应同时定义可移动资产以及需要移除和补全的场景内容。据此,场景观测为耦合场景构建提供共享证据,后者创建已注册的资产和补全的背景高斯,用于交互式环境中的仿真器驱动渲染。这种耦合在保持未编辑高斯的同时,对齐了视觉和物理状态。在50个ScanNet++场景上,基于证据的选择和注册重试在固定保留率下将20毫米处的匹配F1从0.336提升至0.383。进一步测试证明了资产的可执行性、相对于单生成器基线的操作增益以及转换的视觉成本。这些结果共同表明,ϕ-RIE实现了交互式场景转换。

英文摘要

3D Gaussian Splatting (3DGS) can reconstruct a captured scene photorealistically, but the resulting representation does not by itself support physical interaction. Robot simulation instead requires object-level change, \textit{i.e.}, objects must move independently, make contact, and reveal previously occluded surroundings. This gap arises because object appearance may remain entangled with the background, while hidden object geometry and occluded background content may be unobserved. To address this challenge, we present ϕ-RIE, a Gaussian-native pipeline that converts selected objects into movable simulator assets while preserving the remaining reconstruction. Our key observation is that asset construction and source removal should be coupled, \textit{i.e.}, one object identity should define the movable asset and the scene content to remove and complete. Accordingly, Scene Observation supplies shared evidence to Coupled Scene Construction, which creates registered assets and completed background Gaussians for simulator-driven rendering in an Interactive Environment. This coupling preserves unedited Gaussians while aligning visual and physical state. On 50 ScanNet++ scenes, evidence-based selection and registration retry increase matched F1 at 20\,mm from 0.336 to 0.383 at fixed retention. Further tests demonstrate asset executability, manipulation gains over a single-generator baseline, and the visual cost of conversion. Together, these results demonstrate that \name\ enables interactive scene conversion.

Comments8 pages, 6 figures

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑