具有自适应干预机制的机器人门控交互式模仿学习
Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism
AI总结：
针对交互式模仿学习中人类监督负担过重的问题，提出机器人门控的AIM算法，用代理Q函数学习自适应干预请求准则，显著降低接管成本并提升学习效率。
摘要：
交互式模仿学习（IIL）允许智能体通过人类干预获得期望行为，但当前方法对人类监督者提出了很高的认知需求。我们提出自适应干预机制（AIM），这是一种新颖的机器人门控IIL算法，可学习用于请求人类演示的自适应准则。AIM利用代理Q函数来模拟人类干预规则，并根据智能体与人类动作之间的一致性调整干预请求。通过在智能体偏离专家时赋予高Q值，并在智能体变得熟练时降低这些值，代理Q函数使智能体能够评估与专家的实时一致性，并在需要时请求协助。我们的专家在环实验表明，AIM在连续和离散控制任务中都显著减少了专家监控工作量。与基于不确定性的基线Thrifty-DAgger相比，我们的方法在人类接管成本和学习效率方面实现了40%的提升。此外，AIM有效识别需要专家协助的安全关键状态，从而收集更高质量的专家演示，并减少所需的总体专家数据和环境交互。代码和演示视频可在https://github.com/metadriverse/AIM获取。
英文摘要：
Interactive Imitation Learning (IIL) allows agents to acquire desired behaviors through human interventions, but current methods impose high cognitive demands on human supervisors. We propose the Adaptive Intervention Mechanism (AIM), a novel robot-gated IIL algorithm that learns an adaptive criterion for requesting human demonstrations. AIM utilizes a proxy Q-function to mimic the human intervention rule and adjusts intervention requests based on the alignment between agent and human actions. By assigning high Q-values when the agent deviates from the expert and decreasing these values as the agent becomes proficient, the proxy Q-function enables the agent to assess the real-time alignment with the expert and request assistance when needed. Our expert-in-the-loop experiments reveal that AIM significantly reduces expert monitoring efforts in both continuous and discrete control tasks. Compared to the uncertainty-based baseline Thrifty-DAgger, our method achieves a 40% improvement in terms of human take-over cost and learning efficiency. Furthermore, AIM effectively identifies safety-critical states for expert assistance, thereby collecting higher-quality expert demonstrations and reducing overall expert data and environment interactions needed. Code and demo video are available at https://github.com/metadriverse/AIM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
