为灵巧操作扩展跨实体世界模型
Scaling Cross-Embodiment World Models for Dexterous Manipulation
AI总结：
研究跨实体学习中不同形态机器人数据共享和控制转移问题，提出将人类和机器人手表示为3D粒子集，在随机交互数据上训练基于图的世界模型并与模型预测控制集成，实验表明该方法能提高泛化能力、有效结合数据并实现不同机器人手的控制。
摘要：
跨实体学习旨在构建能从不同形态中学习并操作的通用机器人，但运动学和动作空间的差异阻碍了数据共享和控制转移。我们提出疑问：尽管存在这些差异，跨实体可共享何种结构？我们认为它们引发的物理交互可在共享几何空间中建模，使世界模型为学习和控制提供通用接口。为此，将人类和机器人手表示为3D粒子集，把动作定义为末端执行器粒子位移场。在来自不同模拟机器人手和真实人类手的随机交互数据上训练基于图的世界模型，并与模型预测控制集成以部署到新硬件上。在刚性和可变形操作实验中有三个发现：增加训练实体的多样性可提高对未见手的泛化能力；适当结合模拟和现实世界数据比单独使用任何一种数据来源效果更好；相同的学习模型能对具有不同运动学和自由度的机器人手进行有效控制。这些结果表明基于粒子的世界模型是用于跨实体学习和控制的通用接口。
英文摘要：
Cross-embodiment learning seeks to build generalist robots that learn from and operate across diverse morphologies, but differences in kinematics and action spaces hinder data sharing and control transfer. We ask: What structure can be shared across embodiments despite these differences? We argue that the physical interactions they induce can be modeled in a shared geometric space, allowing world models to provide a common interface for learning and control. To realize this idea, we represent human and robot hands as sets of 3D particles and define actions as end-effector particle displacement fields. This representation abstracts away embodiment-specific joint spaces while preserving the geometry and motion relevant to physical interaction. We train a graph-based world model on random interaction data from diverse simulated robot hands and real human hands, and integrate it with model-predictive control for deployment on new hardware. Experiments on rigid and deformable manipulation reveal three findings: increasing the diversity of training embodiments improves generalization to unseen hands; appropriately combining simulated and real-world data outperforms either source alone; and the same learned model enables effective control on robotic hands with distinct kinematics and degrees of freedom. These results position particle-based world models as a shared interface for learning from and for heterogeneous embodiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
