面向人形机器人视觉驱动灵巧操作的Sim-to-Real强化学习
Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids
AI总结：
针对人形机器人视觉灵巧操作策略学习难、现有方法成本高的问题，提出含自动现实-仿真调优、通用奖励等模块的sim-to-real强化学习方案，在三项任务上实现高成功率与鲁棒性，验证了该方案的可行性与可扩展性。
摘要：
学习具备泛化能力的机器人操作策略，尤其是针对复杂多指人形机器人的策略，仍是一项重大挑战。现有方法主要依赖大规模数据采集与模仿学习，这类方法成本高昂、耗费人力且难以规模化。Sim-to-real（仿真到现实）强化学习（RL）提供了一种颇具前景的替代方案，但此前大多仅在更简单的基于状态的设置或单手机器人设置中取得成功。如何将其有效拓展至视觉驱动、接触丰富的双手操作任务，仍是一个悬而未决的问题。本文提出了一套实用的sim-to-real强化学习方案，用于训练人形机器人完成三项具有挑战性的灵巧操作任务：抓取与伸展、箱子 lifting 以及双手交接。该方法包含自动化real-to-sim（现实到仿真）调优模块、基于接触与物体目标的通用奖励公式、分治策略蒸馏框架，以及结合模态特定增强的混合物体表示策略。我们在未见过的物体上实现了高成功率，且策略表现出鲁棒的自适应行为——这表明通过sim-to-real强化学习实现视觉驱动的灵巧操作不仅可行，还具备可扩展性，可广泛应用于现实世界的人形机器人操作任务。
英文摘要：
Learning generalizable robot manipulation policies, especially for complex multi-fingered humanoids, remains a significant challenge. Existing approaches primarily rely on extensive data collection and imitation learning, which are expensive, labor-intensive, and difficult to scale. Sim-to-real reinforcement learning (RL) offers a promising alternative, but has mostly succeeded in simpler state-based or single-hand setups. How to effectively extend this to vision-based, contact-rich bimanual manipulation tasks remains an open question. In this paper, we introduce a practical sim-to-real RL recipe that trains a humanoid robot to perform three challenging dexterous manipulation tasks: grasp-and-reach, box lift and bimanual handover. Our method features an automated real-to-sim tuning module, a generalized reward formulation based on contact and object goals, a divide-and-conquer policy distillation framework, and a hybrid object representation strategy with modality-specific augmentation. We demonstrate high success rates on unseen objects and robust, adaptive policy behaviors -- highlighting that vision-based dexterous manipulation via sim-to-real RL is not only viable, but also scalable and broadly applicable to real-world humanoid manipulation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
