从感知到模拟：利用数字亲属进行生成高保真模拟，用于可推广的机器人学习与评估

中文摘要 AI 辅助

在真实世界环境中学习鲁棒的机器人策略需要多样化的数据增强，但扩大真实世界数据收集的成本高，因为需要获取物理资产并重新配置环境。因此，将真实世界场景增强到模拟成为一种实用的增强方法，用于高效学习和评估。我们提出一个生成框架，建立从真实世界全景到高保真模拟场景的生成映射，并通过语义和几何编辑合成多样化的亲属场景。结合高质量物理引擎和真实资产，生成的场景支持交互式操作任务。此外，我们整合多房间拼接技术，构建一致的大规模环境，以支持长周期导航在复杂布局中。实验展示了强的模拟到现实相关性，验证了我们平台的保真度，并表明大幅扩展数据生成能显著提高对未见过场景和物体变化的泛化能力，证明了数字亲属在可推广的机器人学习和评估中的有效性。

英文摘要

Learning robust robot policies in real-world environments requires diverse data augmentation, yet scaling real-world data collection is costly due to the need for acquiring physical assets and reconfiguring environments. Therefore, augmenting real-world scenes into simulation has become a practical augmentation for efficient learning and evaluation. We present a generative framework that establishes a generative real-to-sim mapping from real-world panoramas to high-fidelity simulation scenes, and further synthesize diverse cousin scenes via semantic and geometric editing. Combined with high-quality physics engines and realistic assets, the generated scenes support interactive manipulation tasks. Additionally, we incorporate multi-room stitching to construct consistent large-scale environments for long-horizon navigation across complex layouts. Experiments demonstrate a strong sim-to-real correlation validating our platform's fidelity, and show that extensively scaling up data generation leads to significantly better generalization to unseen scene and object variations, demonstrating the effectiveness of Digital Cousins for generalizable robot learning and evaluation.

发表机构

• Peking University（北京大学）
• Lightwheel

机构由 AI 辅助整理，请以论文原文为准。