如何缓解机器人控制中的分布偏移问题：一种基于离线到在线模仿学习的鲁棒自适应方法
How to Mitigate the Distribution Shift Problem in Robotics Control: A Robust and Adaptive Approach Based on Offline to Online Imitation Learning
AI总结：
提出一种鲁棒离线到自适应在线模仿学习框架，通过离线阶段利用判别器扩展状态-动作覆盖和在线阶段自监督模仿学习，缓解分布偏移问题。
摘要：
模仿学习中的分布偏移是指智能体无法为训练期间未访问的状态规划适当动作的问题。该问题很大程度上归因于专家演示在整个环境中提供的固有狭窄状态-动作覆盖。在本文中，我们提出了一种鲁棒离线到自适应在线模仿学习框架，以终身、多阶段方案处理分布偏移问题。在离线学习阶段，我们利用补充演示通过判别器有效训练策略，从而拓宽策略的状态-动作覆盖，增强策略对分布偏移的鲁棒性。在后续的在线推理阶段，我们的框架检测分布偏移的发生，并从在线经验中进行自监督模仿学习，使策略适应在线环境。通过在MuJoCo环境中的广泛评估，我们证明我们的方法在分布偏移的鲁棒性和对在线环境的适应性能方面优于基线算法，这表明我们的框架在对抗分布偏移方面具有优越性能。
英文摘要：
Distribution shift in imitation learning refers to the problem that the agent cannot plan proper actions for a state that has not been visited during the training. This problem can be largely attributed to the inherently narrow state-action coverage provided by expert demonstrations over the full environment. In this paper, we propose a robust offline to adaptive online imitation learning framework that handles the distribution shift problem in a lifelong, multi-phase scheme. In the offline learning phase, we leverage supplementary demonstrations to broaden the state-action coverage of the policy by utilizing a discriminator to effectively train the policy with supplementary demonstrations, thereby enhancing the robustness of the policy to distribution shift. In the subsequent online inference phase, our framework detects the occurrence of distribution shift and conducts self-supervised imitation learning from online experiences to adapt the policy to the online environments. Through extensive evaluations in MuJoCo environments, we demonstrate that our method exhibits better robustness to distribution shift and better adaptation performance to online environments than the baseline algorithms, which indicates superior performance of our framework against the distribution shift.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
