arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.00399cs.AI
收藏

在强化学习中使用记忆编辑来学习用户定义的子目标

Learning user-defined sub-goals using memory editing in reinforcement learning

Rutgers University, The State University of New Jersey(罗格斯大学,新泽西州立大学)

GyeongTaek Lee

摘要:

强化学习(RL)的目的是让智能体实现最终目标。大多数RL研究都集中在提高学习效率,以更快地实现最终目标。然而,强化学习模型在达到最终目标的过程中很难修改中间路线。也就是说,在现有的研究中,智能体不能被控制以实现其他子目标。如果智能体能够在到达目的地的过程中遍历子目标,强化学习就可以在各个领域得到应用和研究。在这项研究中,我提出了一种方法来实现用户定义的子目标,以及使用记忆编辑的最终目标。执行内存编辑以生成各种子目标并向代理提供额外奖励。此外,子目标是从最终目标中单独学习的。我在测试环境中设置了两个简单的环境和各种场景。结果,智能体几乎成功地通过了子目标以及控制下的最终目标。此外,智能体能够被诱导间接访问环境中的新状态。我希望这种方法可以用于需要在各种场景中控制代理的领域。

英文摘要:

The aim of reinforcement learning (RL) is to allow the agent to achieve the final goal. Most RL studies have focused on improving the efficiency of learning to achieve the final goal faster. However, the RL model is very difficult to modify an intermediate route in the process of reaching the final goal. That is, the agent cannot be under control to achieve other sub-goals in the existing studies. If the agent can go through the sub-goals on the way to the destination, the RL can be applied and studied in various fields. In this study, I propose a methodology to achieve the user-defined sub-goals as well as the final goal using memory editing. The memory editing is performed to generate various sub-goals and give an additional reward to the agent. In addition, the sub-goals are separately learned from the final goal. I set two simple environments and various scenarios in the test environments. As a result, the agent almost successfully passed the sub-goals as well as the final goal under control. Moreover, the agent was able to be induced to visit the novel state indirectly in the environments. I expect that this methodology can be used in the fields that need to control the agent in a variety of scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑