arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.02566cs.LGcs.AI
收藏

迷宫求解 Transformer 中的结构化世界表征

Structured World Representations in Maze-Solving Transformers

Colorado School of Mines(科罗拉多矿业学院);Imperial College London(伦敦帝国理工学院);National Institute of Informatics(信息学研究所)
Michael Igorevich Ivanitskiy, Alex F. Spies, Tilman Räuker, Guillaume Corlouer, Chris Mathwin, Lucia Quirke, Can Rager, Rusheb Shah, Dan Valentine, Cecilia Dini… 展开作者

Michael Igorevich Ivanitskiy, Alex F. Spies, Tilman Räuker, Guillaume Corlouer, Chris Mathwin, Lucia Quirke, Can Rager, Rusheb Shah, Dan Valentine, Cecilia Diniz Behn, Katsumi Inoue, Samy Wu Fung

AI总结:

本文研究求解迷宫的小型 Transformer,发现其内部会形成迷宫拓扑与有效路径的结构化表征,并通过单 token 线性解码和邻接头分析加以验证。

摘要:

Transformer 模型支撑了近期实际机器学习应用中的许多进展,但研究人员仍难以理解其内部行为。鉴于这些模型的规模和复杂性,全面认识其内部工作机制仍是一项重大挑战。为此,我们着手在一个更易处理的场景中理解小型 Transformer 模型:求解迷宫。在本研究中,我们关注这些模型形成的抽象,发现了迷宫拓扑和有效路径的结构化内部表征持续涌现的证据。我们证明,仅对单个 token 的残差流进行线性解码,就能忠实重建整个迷宫。我们还发现,各个 token 学到的嵌入具有空间结构。此外,我们通过识别被称为“邻接头”(adjacency heads)的注意力头,进一步揭示路径遵循机制的电路;这些注意力头与寻找有效的后续 token 有关。

英文摘要:

Transformer models underpin many recent advances in practical machine learning applications, yet understanding their internal behavior continues to elude researchers. Given the size and complexity of these models, forming a comprehensive picture of their inner workings remains a significant challenge. To this end, we set out to understand small transformer models in a more tractable setting: that of solving mazes. In this work, we focus on the abstractions formed by these models and find evidence for the consistent emergence of structured internal representations of maze topology and valid paths. We demonstrate this by showing that the residual stream of only a single token can be linearly decoded to faithfully reconstruct the entire maze. We also find that the learned embeddings of individual tokens have spatial structure. Furthermore, we take steps towards deciphering the circuity of path-following by identifying attention heads (dubbed $\textit{adjacency heads}$), which are implicated in finding valid subsequent tokens.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑