arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.04327cs.ROcs.AIcs.LG
收藏

基于记忆的学习方法解决状态-动作约束的任务

Using Memory-Based Learning to Solve Tasks with State-Action Constraints

Carnegie Mellon University(卡内基梅隆大学)

Mrinal Verghese, Chris Atkeson

摘要:

可能的动作集不连续地依赖于状态的任务对当前的强化学习算法构成了重大挑战。例如,一扇锁着的门必须先打开,然后转动把手,门才能打开。这些任务的连续性使得获得最终奖励变得困难,并且使用连续学习值(如权重)而不是离散符号在任务变量之间传输信息可能效率低下。我们的关键见解是,象征性地行动和思考的代理通常在处理这些任务时更有效。我们提出了一种基于记忆的学习方法,利用这些任务中的约束和动作的时间顺序的象征性,以快速获取和传递高层次的信息。我们评估了基于记忆的学习在真实和模拟任务上的性能,这些任务在状态和动作之间具有近似不连续的约束,并表明我们的方法学习解决这些任务的速度比基于模型和无模型的深度强化学习方法快一个数量级。

英文摘要:

Tasks where the set of possible actions depend discontinuously on the state pose a significant challenge for current reinforcement learning algorithms. For example, a locked door must be first unlocked, and then the handle turned before the door can be opened. The sequential nature of these tasks makes obtaining final rewards difficult, and transferring information between task variants using continuous learned values such as weights rather than discrete symbols can be inefficient. Our key insight is that agents that act and think symbolically are often more effective in dealing with these tasks. We propose a memory-based learning approach that leverages the symbolic nature of constraints and temporal ordering of actions in these tasks to quickly acquire and transfer high-level information. We evaluate the performance of memory-based learning on both real and simulated tasks with approximately discontinuous constraints between states and actions, and show our method learns to solve these tasks an order of magnitude faster than both model-based and model-free deep reinforcement learning methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑