Policy Decorator：面向大型策略模型的模型无关在线优化
Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
AI总结：
本文提出Policy Decorator，一种模型无关的残差策略方法，通过在线交互优化大型模仿学习模型，在ManiSkill和Adroit基准上显著提升了策略性能并保持了平滑运动。
摘要：
机器人学习的最新进展利用大型模型和大量演示进行模仿学习，以开发有效的策略。然而，这些模型通常受限于演示的数量、质量和多样性。本文探讨通过与环境的在线交互来改进离线训练的模仿学习模型。我们提出了Policy Decorator，它利用模型无关的残差策略在在线交互期间优化大型模仿学习模型。通过实施受控的探索策略，Policy Decorator能够实现稳定且样本高效的在线学习。我们的评估涵盖了两个基准ManiSkill和Adroit上的八个任务，并涉及两个最先进的模仿学习模型（Behavior Transformer和Diffusion Policy）。结果表明，Policy Decorator有效改进了离线训练的策略，并保持了模仿学习模型的平滑运动，避免了纯RL策略的 erratic 行为。有关视频，请参阅我们的项目页面（https://policydecorator.github.io）。
英文摘要：
Recent advancements in robot learning have used imitation learning with large models and extensive demonstrations to develop effective policies. However, these models are often limited by the quantity, quality, and diversity of demonstrations. This paper explores improving offline-trained imitation learning models through online interactions with the environment. We introduce Policy Decorator, which uses a model-agnostic residual policy to refine large imitation learning models during online interactions. By implementing controlled exploration strategies, Policy Decorator enables stable, sample-efficient online learning. Our evaluation spans eight tasks across two benchmarks-ManiSkill and Adroit-and involves two state-of-the-art imitation learning models (Behavior Transformer and Diffusion Policy). The results show Policy Decorator effectively improves the offline-trained policies and preserves the smooth motion of imitation learning models, avoiding the erratic behaviors of pure RL policies. See our project page (https://policydecorator.github.io) for videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
