Dex4D：一种任务无关的点轨迹策略用于仿真到现实的灵巧操作
Dex4D: Task-Agnostic Point Track Policy for Zero-Shot Dexterous Manipulation
AI总结：
Dex4D通过仿真学习任务无关的点轨迹策略，实现零样本部署的灵巧操作任务。
摘要：
学习能够完成大量日常任务的通用策略仍然是灵巧操作中的一个开放性挑战。特别是，通过现实世界遥控收集大规模操作数据是昂贵且难以扩展的。虽然在仿真中学习提供了一个可行的替代方案，但设计多个任务特定的环境和奖励进行训练同样具有挑战性。我们提出了Dex4D，一种框架，它利用仿真来学习任务无关的灵巧技能，这些技能可以灵活地重新组合以执行多样化的现实世界操作任务。具体来说，Dex4D学习了一个领域无关的3D点轨迹条件策略，能够操控任何物体到任何期望的姿态。我们通过成千上万的具有多样化姿态配置的对象在仿真中训练这个'AnyPose-to-AnyPose'策略，覆盖了广泛范围的机器人-物体交互空间，可以在测试时进行组合。在部署时，该策略可以无需微调直接转移到现实世界任务，只需通过提示提取自生成视频中的目标物体中心点轨迹即可。在执行过程中，Dex4D使用在线点轨迹进行闭环感知和控制。在仿真和真实机器人上的大量实验表明，我们的方法能够实现零样本部署以执行多样化的灵巧操作任务，并在先前基线方法上产生一致的改进。此外，我们展示了对新对象、场景布局、背景和轨迹的强大泛化能力，突显了所提出框架的鲁棒性和可扩展性。
英文摘要：
Learning generalist policies capable of accomplishing a wide range of everyday tasks remains an open challenge in dexterous manipulation. In particular, collecting large-scale manipulation data via real-world teleoperation is expensive and difficult to scale. While learning in simulation provides a feasible alternative, designing multiple task-specific environments and rewards for training is similarly challenging. We propose Dex4D, a framework that instead leverages simulation for learning task-agnostic dexterous skills that can be flexibly recomposed to perform diverse real-world manipulation tasks. Specifically, Dex4D learns a domain-agnostic 3D point track conditioned policy capable of transforming a rigid object from a current pose to a target pose. We train this 'Anypose-to-Anypose' policy in simulation across thousands of objects with diverse pose configurations, covering a broad space of robot-object interactions that can be composed at test time. At deployment, this policy can be zero-shot transferred to real-world tasks without finetuning, simply by prompting it with desired object-centric point tracks extracted from generated videos. During execution, Dex4D uses online point tracking for closed-loop perception and control. Extensive experiments in simulation and on real robots show that our method enables zero-shot deployment for diverse dexterous manipulation tasks and yields consistent improvements over prior baselines. Furthermore, we demonstrate strong generalization to novel objects, scene layouts, backgrounds, and trajectories, highlighting the robustness and scalability of the proposed framework. Project page: https://dex4d.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
