DexPIE：基于真实世界经验的稳定灵巧策略改进
DexPIE: Stable Dexterous Policy Improvement from Real-World Experience
AI总结：
提出DexPIE后训练框架，通过灵巧手适配干预系统、多阶段DAgger数据收集、相对动作空间异步推理和连续最优性指标条件化，在三个真实灵巧操作任务上成功率提升37%。
摘要：
灵巧操作因其高维动作空间和复杂的接触动力学，给模仿学习带来了巨大挑战。纯粹从演示中训练的策略在部署时常常遭受复合误差，并且需要大量专家数据才能达到可靠性能。为了超越演示数据的局限性，本文提出DexPIE，一个通过真实世界部署收集的经验来改进灵巧策略的后训练框架。首先，DexPIE通过灵巧手适配的干预系统和跨初始与中间任务阶段的多阶段DAgger式数据收集，实现了有效的探索覆盖，为准确的策略评估提供了可靠的监督。为了减少后训练 rollout 与演示数据之间的时间噪声，我们引入了相对动作空间中的异步推理，这能更好地将 rollout 数据与演示行为对齐，并允许评论家学习由更一致的基础策略诱导的值函数。最后，DexPIE通过对连续最优性指标进行条件化来改进策略，使策略能够以更细粒度的方式利用数据质量。在三个具有挑战性的真实世界灵巧操作任务中，DexPIE相比基于演示的参考策略实现了37%的成功率提升，优于所有基线方法，并展现出更强的鲁棒性。源代码和数据集将公开提供。
英文摘要：
Dexterous manipulation presents substantial challenges for imitation learning due to its high-dimensional action space and complex contact-rich dynamics. Policies trained purely from demonstrations often suffer from compounding errors during deployment and require large amounts of expert data to achieve reliable performance. To move beyond the limitations of demonstration data, in this work, we propose DexPIE, a post-training framework for dexterous policy improvement from experience collected through real-world deployment. First, DexPIE enables effective exploration coverage through a dexterous-hand-adapted intervention system and multi-stage DAgger-style data collection across initial and intermediate task stages. Meanwhile, we enhance consistency between training and inference to reduce the distribution shift between rollouts and demonstration data, better aligning rollout behavior with demonstrations, allowing the critic to learn a value function induced by a more consistent underlying policy. Together, these components provide reliable supervision for policy evaluation. Finally, DexPIE improves the policy through conditioning on a continuous optimality indicator, allowing the policy to leverage the quality of data in a more fine-grained manner. Across three challenging real-world dexterous manipulation tasks, DexPIE achieves a 37.3% improvement in success rate over the demonstration-based reference policy, outperforming all baseline methods and demonstrating stronger robustness. The source code and dataset will be made publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
