ManiSkill3：面向可泛化具身智能的GPU并行机器人仿真与渲染
ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI
AI总结：
本文提出并开源了ManiSkill3，一个支持GPU并行化的高速机器人仿真与渲染系统，通过富接触物理和全面的任务环境，显著提升了可泛化具身AI的训练效率与sim2real能力。
摘要：
仿真为机器人学习实现了前所未有的计算可扩展方法。然而，许多现有仿真框架通常仅支持狭窄范围的场景/任务，并且缺乏对扩展可泛化机器人技术和sim2real至关重要的特性。我们引入并开源了ManiSkill3，这是最快的、支持状态-视觉的GPU并行机器人仿真器，具备针对可泛化操作的富接触物理特性。ManiSkill3支持多方面的GPU并行化，包括仿真+渲染、异构仿真、点云/体素视觉输入等。与其他平台相比，ManiSkill3上的带渲染仿真运行速度可快10-1000倍，GPU内存使用量减少2-3倍，在基准测试环境中达到30000+ FPS，这得益于系统中极低的python/pytorch开销、GPU上的仿真以及SAPIEN并行渲染系统的使用。过去需要数小时训练的任务现在只需几分钟。我们进一步提供了最全面的GPU并行环境/任务集，涵盖12个不同领域，包括但不限于移动操作（如绘画任务）、人形机器人，以及由艺术家设计的真实场景或真实世界数字孪生中的灵巧操作。此外，还提供了来自运动规划、RL和遥操作的数百万个演示帧。ManiSkill3还提供了一套全面的基线，涵盖流行的RL和-learning-from-demonstrations算法。
英文摘要：
Simulation has enabled unprecedented compute-scalable approaches to robot learning. However, many existing simulation frameworks typically support a narrow range of scenes/tasks and lack features critical for scaling generalizable robotics and sim2real. We introduce and open source ManiSkill3, the fastest state-visual GPU parallelized robotics simulator with contact-rich physics targeting generalizable manipulation. ManiSkill3 supports GPU parallelization of many aspects including simulation+rendering, heterogeneous simulation, pointclouds/voxels visual input, and more. Simulation with rendering on ManiSkill3 can run 10-1000x faster with 2-3x less GPU memory usage than other platforms, achieving up to 30,000+ FPS in benchmarked environments due to minimal python/pytorch overhead in the system, simulation on the GPU, and the use of the SAPIEN parallel rendering system. Tasks that used to take hours to train can now take minutes. We further provide the most comprehensive range of GPU parallelized environments/tasks spanning 12 distinct domains including but not limited to mobile manipulation for tasks such as drawing, humanoids, and dextrous manipulation in realistic scenes designed by artists or real-world digital twins. In addition, millions of demonstration frames are provided from motion planning, RL, and teleoperation. ManiSkill3 also provides a comprehensive set of baselines that span popular RL and learning-from-demonstrations algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
