arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.01684cs.LGcs.AI
收藏

面向无重置强化学习的智能切换

Intelligent Switching for Reset-Free RL

Mila(米拉计算研究所);Université de Montréal(蒙特利尔大学);Dalhousie University(达尔豪斯大学);École Polytechnique de Montréal(蒙特利尔综合理工学院)

Darshan Patil, Janarthanan Rajendran, Glen Berseth, Sarath Chandar

AI总结:

本文提出RISC算法,通过基于智能体置信度的智能切换机制协调前向与后向智能体,解决无重置强化学习中转换时机问题,并在多个挑战性环境中取得最优性能。

摘要:

在现实世界中,训练智能体所需的强情节重置机制在仿真中不可用。重置假设限制了强化学习在现实世界中的潜力,因为为智能体提供重置通常需要创建额外的手工机制或人工干预。近期工作旨在通过构建第二个(后向)智能体将前向智能体返回初始状态,从而利用学习到的重置来训练(前向)智能体。我们发现这两个智能体之间转换的终止条件和时机对算法成功至关重要。基于此,我们提出了一种新算法——带智能切换控制器的无重置强化学习(RISC),该算法根据智能体对实现当前目标的置信度在两个智能体之间进行智能切换。我们的新方法在多个具有挑战性的无重置强化学习环境中取得了最先进的性能。

英文摘要:

In the real world, the strong episode resetting mechanisms that are needed to train agents in simulation are unavailable. The \textit{resetting} assumption limits the potential of reinforcement learning in the real world, as providing resets to an agent usually requires the creation of additional handcrafted mechanisms or human interventions. Recent work aims to train agents (\textit{forward}) with learned resets by constructing a second (\textit{backward}) agent that returns the forward agent to the initial state. We find that the termination and timing of the transitions between these two agents are crucial for algorithm success. With this in mind, we create a new algorithm, Reset Free RL with Intelligently Switching Controller (RISC) which intelligently switches between the two agents based on the agent's confidence in achieving its current goal. Our new method achieves state-of-the-art performance on several challenging environments for reset-free RL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑