arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.15421cs.LGcs.AI
收藏

在玩Sokoban的循环神经网络中的规划

Planning in a recurrent neural network that plays Sokoban

Berkeley(伯克利);Jane Street

Mohammad Taufeeque, Philip Quirke, Maximilian Li, Chris Cundy, Aaron David Tucker, Adam Gleave, Adrià Garriga-Alonso

AI总结:

分析了在Sokoban上训练的RNN,发现其具备提前约50步预测动作的因果规划表示,并通过“踱步”循环增加计算,能泛化至更大谜题。

摘要:

规划对于解决复杂任务至关重要,然而神经网络中规划背后的内部机制仍知之甚少。基于先前的工作,我们分析了一个在Sokoban上训练的循环神经网络(RNN),Sokoban是一个需要序列化、不可逆决策的挑战性谜题。我们发现RNN具有因果规划表示,能够提前约50步预测其未来动作。表示的规划的质量和长度在最初几步内会增加。我们揭示了一个令人惊讶的行为:RNN在关卡开始时以循环方式“踱步”,为自己提供额外计算,并表明该行为是由训练激励的。利用这些见解,我们将训练好的RNN扩展到显著更大的、分布外的Sokoban谜题,展示了超越训练机制的稳健表示。我们开源了模型和代码,并相信神经网络的有趣行为使其成为加深我们对学习规划理解的优秀模式生物。

英文摘要:

Planning is essential for solving complex tasks, yet the internal mechanisms underlying planning in neural networks remain poorly understood. Building on prior work, we analyze a recurrent neural network (RNN) trained on Sokoban, a challenging puzzle requiring sequential, irreversible decisions. We find that the RNN has a causal plan representation which predicts its future actions about 50 steps in advance. The quality and length of the represented plan increases over the first few steps. We uncover a surprising behavior: the RNN "paces" in cycles to give itself extra computation at the start of a level, and show that this behavior is incentivized by training. Leveraging these insights, we extend the trained RNN to significantly larger, out-of-distribution Sokoban puzzles, demonstrating robust representations beyond the training regime. We open-source our model and code, and believe the neural network's interesting behavior makes it an excellent model organism to deepen our understanding of learned planning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑