用于具身AI和机器人模拟的3D生成:综述
3D Generation for Embodied AI and Robotic Simulation: A Survey
AI总结:
本文综述了用于具身AI的3D生成,探讨了其在数据生成、模拟环境构建和Sim2Real桥梁中的作用,指出领域正从视觉真实转向交互准备,并识别了物理标注有限、几何质量与物理有效性差距等主要瓶颈。
摘要:
具身AI和机器人系统日益依赖可扩展、多样且具有物理基础的3D内容用于基于模拟的训练和现实部署。尽管3D生成模型迅速发展,但具身应用对视觉真实性的要求远超其能力:生成的对象必须具备运动学结构和材料属性,场景必须支持交互和任务执行,且生成的内容必须弥合模拟与现实之间的差距。本文综述了用于具身AI的3D生成,围绕其在具身系统中扮演的三个角色组织文献。在数据生成中,3D生成产生用于模拟的物体和资产,包括可动、具有物理基础和可变形的内容以供后续交互;在模拟环境中,它构建交互性和任务导向的世界,涵盖结构感知、可控和代理场景生成;在Sim2Real桥梁中,它支持数字双重建、数据增强和合成演示以支持下游机器人学习和现实转移。我们还表明,该领域正从视觉真实转向交互准备,并识别了主要瓶颈,包括物理标注有限、几何质量与物理有效性差距、评估碎片化以及持续的模拟到现实差距,这些必须解决才能使3D生成成为具身智能的可靠基础。我们的项目页面位于https://3dgen4robot.github.io。
英文摘要:
Embodied AI and robotic systems increasingly depend on scalable, diverse, and physically grounded 3D content for simulation-based training and real-world deployment. While 3D generative modeling has advanced rapidly, embodied applications impose requirements far beyond visual realism: generated objects must carry kinematic structure and material properties, scenes must support interaction and task execution, and the resulting content must bridge the gap between simulation and reality. This survey reviews 3D generation for embodied AI and organizes the literature around three roles that 3D generation plays in embodied systems. In Data Generator, 3D generation produces simulation-ready objects and assets, including articulated, physically grounded, and deformable content for downstream interaction; in Simulation Environments, it constructs interactive and task-oriented worlds, spanning structure-aware, controllable, and agentic scene generation; and in Sim2Real Bridge, it supports digital twin reconstruction, data augmentation, and synthetic demonstrations for downstream robot learning and real-world transfer. We also show that the field is shifting from visual realism toward interaction readiness, and we identify the main bottlenecks, including limited physical annotations, the gap between geometric quality and physical validity, fragmented evaluation, and the persistent sim-to-real divide, that must be addressed for 3D generation to become a dependable foundation for embodied intelligence. Our project page is at https://3dgen4robot.github.io.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。