基于验证器的免更新在线策略引导
Update-Free On-Policy Steering via Verifiers
AI总结：
提出UF-OPS方法，利用策略评估中的验证器函数引导基础策略选择高成功概率动作，无需更新参数即可提升黑箱扩散策略性能，在5个真实任务中平均成功率提升49%。
摘要：
近年来，行为克隆（BC）已成为从人类演示中学习操作的最流行方法之一。尽管取得了成功，但BC策略通常脆弱且难以进行精确操作。为了克服这些问题，我们提出了UF-OPS，一种免更新的在线策略引导方法，使机器人能够预测其动作的成功可能性，并在执行时调整策略。我们通过使用在策略初始评估期间获得的策略回滚数据来训练验证器函数来实现这一点。这些验证器随后用于将基础策略引导向具有更高成功可能性的动作。我们的方法在不改变基础参数的情况下提高了黑箱扩散策略的性能，使其轻量且灵活。我们展示了来自仿真和真实世界数据的结果，并在5个真实任务中实现了比基础策略平均49%的成功率提升。
英文摘要：
In recent years, Behavior Cloning (BC) has become one of the most prevalent methods for learning manipulation from human demonstrations. Despite their successes, BC policies are often brittle and struggle with precise manipulation. To overcome these issues, we propose UF-OPS, an Update-Free On-Policy Steering method that enables the robot to predict the success likelihood of its actions and adapt its strategy at execution time. We accomplish this by training verifier functions using policy rollout data obtained during an initial evaluation of the policy. These verifiers are subsequently used to steer the base policy toward actions with a higher likelihood of success. Our method improves the performance of black-box diffusion policies, without changing the base parameters, making it lightweight and flexible. We present results from both simulation and real-world data and achieve an average 49% improvement in success rate over the base policy across 5 real tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
