HERO: 学习人形机器人的末端执行器控制用于视觉全身体对象抓取
HERO: Learning Humanoid End-Effector Control for Visual Whole-Body Open-Vocabulary Object Grasping
AI总结：
该研究提出HERO方法，通过结合大视觉模型和模拟训练，实现了视觉全身体对象抓取任务中末端执行器的高精度控制和场景理解，显著提升了抓取精度和泛化能力。
摘要：
视觉定位和操作任意真实场景中的物体需要精确的末端执行器（EE）控制和从视觉输入（如RGB-D图像）中获得的可泛化场景理解。现有的模仿和仿真到现实的方法通过单体端到端学习同时学习这两个方面，因此难以扩展。在本工作中，我们利用最适合每个问题的工具——大视觉模型用于可泛化的场景理解和模拟训练用于精确的末端执行器控制，从而得到一个整体模块化的定位和操作系统，表现出强大的泛化能力。我们的核心技术创新是HERO，一个通过结合经典机器人学和机器学习实现的准确残差感知末端执行器跟踪策略。它利用a）逆运动学将残差末端执行器目标转换为参考轨迹，b）一个学习的神经前向模型用于准确的前向运动学，以及c）目标调整和重新规划。这些创新共同将末端执行器跟踪误差减少到2.44厘米，优于最强的先前方法5.5倍。我们的整体系统在多样化的现实环境中运行，从办公室到咖啡馆，机器人能够可靠地抓取各种日常物体（如杯子、苹果、玩具）在高度从43厘米到92厘米的表面上。系统性的模块化和端到端测试验证了我们提出设计的有效性。我们相信我们的进展为训练人形机器人与日常物体互动开辟了新途径。
英文摘要：
Visual loco-manipulation of arbitrary in-the-wild objects requires accurate end-effector (EE) control and a generalizable understanding of the scene from visual inputs (eg, RGB-D images). Existing imitation and sim2real methods jointly learn both these aspects via monolithic end-to-end learning and are thus hard to scale. In this work, we bring to bear the best tools for each of these problems -- large vision models for generalizable scene understanding and simulated training for accurate EE control -- leading to an overall modular loco-manipulation system that exhibits strong generalization. Our core technical innovation is HERO, an accurate residual-aware EE tracking policy made possible by combining classical robotics with machine learning. It uses a) inverse kinematics to convert residual end-effector targets into reference trajectories, b) a learned neural forward model for accurate forward kinematics, and c) goal adjustment and replanning. Together, these innovations reduce the end-effector tracking error to 2.44cm, outperforming the strongest prior method by 5.5x. Our overall system operates in diverse real-world environments, from offices to coffee shops, where the robot reliably grasps various everyday objects (eg, mugs, apples, toys) on surfaces ranging from 43cm to 92cm in height. Systematic modular and end-to-end tests demonstrate the effectiveness of our proposed design. We believe our advances open up new ways of training humanoids to interact with daily objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
