Mirage2Matter: 从视频构建一个物理基础的高斯世界模型
Mirage2Matter: A Physically Grounded Gaussian World Model from Video
AI总结：
Mirage2Matter通过视频重建物理基础的世界模型，利用3D高斯点云和生成模型实现高保真具身训练数据生成，提升具身智能的可扩展性和实用性。
摘要：
具身智能的可扩展性本质上受到真实世界交互数据稀缺的限制。虽然模拟平台提供了一个有前景的替代方案，但现有方法往往在视觉和物理上与真实环境存在显著差距，并且依赖于昂贵的传感器、精确的机器人校准或深度测量，限制了其在大规模应用中的实用性。我们提出了Simulate Anything，一个基于图形的世界建模和模拟框架，它能够仅使用多视角环境视频和现成资产高效地生成高保真的具身训练数据。我们的方法利用3D高斯点云（3DGS）将真实世界环境重建为逼真的场景表示，无缝捕捉视频中的细粒度几何和外观。然后，我们利用生成模型恢复出一个物理上真实的表示，并通过一个精度校准目标将其整合到模拟环境中，从而在重建的场景和真实世界之间实现精确的尺度对齐。这些组件共同提供了一个统一的、可编辑的、物理基础的世界模型。在我们的模拟数据上训练的视觉语言动作（VLA）模型在下游任务上实现了强大的零样本性能，其表现与使用真实世界数据获得的结果相当，甚至更好，突显了基于重建的世界建模在可扩展和实用的具身智能训练中的潜力。
英文摘要：
The scalability of embodied intelligence is fundamentally constrained by the scarcity of real-world interaction data. While simulation platforms provide a promising alternative, existing approaches often suffer from a substantial visual and physical gap to real environments and rely on expensive sensors, precise robot calibration, or depth measurements, limiting their practicality at scale. We present Simulate Anything, a graphics-driven world modeling and simulation framework that enables efficient generation of high-fidelity embodied training data using only multi-view environment videos and off-the-shelf assets. Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS), seamlessly capturing fine-grained geometry and appearance from video. We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target, enabling accurate scale alignment between the reconstructed scene and the real world. Together, these components provide a unified, editable, and physically grounded world model. Vision Language Action (VLA) models trained on our simulated data achieve strong zero-shot performance on downstream tasks, matching or even surpassing results obtained with real-world data, highlighting the potential of reconstruction-driven world modeling for scalable and practical embodied intelligence training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
