GPT如何逐层学习
How GPT learns layer by layer
AI总结：
本研究以OthelloGPT为受控测试平台，通过对比稀疏自编码器（SAEs）与线性探针，揭示GPT模型逐层学习棋盘状态与棋局特征的规律，为理解大语言模型的内部表征构建了研究框架。
摘要：
大语言模型（LLMs）在语言处理、策略游戏和推理等任务中表现出色，但难以构建对智能体自适应决策至关重要的可泛化内部表征。智能体要有效应对复杂环境，必须构建可靠的世界模型。尽管LLMs在特定基准测试中表现优异，但它们往往难以泛化，导致表征脆弱，限制了其在现实场景中的有效性。理解LLMs如何构建内部世界模型，是开发能够在不同任务中实现一致、自适应行为的智能体的关键。我们将OthelloGPT——一个在奥赛罗棋对弈数据上训练的基于GPT的模型——作为研究表征学习的受控测试平台进行分析。尽管OthelloGPT仅通过随机合法走子的下一词元预测任务训练，但它在棋盘状态和棋局理解上呈现出有意义的逐层演进规律：早期层捕捉棋盘边界等静态属性，更深层则反映棋子的动态变化。为解释这些表征，我们将稀疏自编码器（Sparse Autoencoders, SAEs）与线性探针进行对比，发现SAEs能对组合特征提供更鲁棒、解耦的洞察，而线性探针主要检测对分类有用的特征。我们利用SAEs解码与棋子颜色和棋子稳定性相关的特征——棋子稳定性是此前未被研究的特征，反映了棋盘控制、长期规划等复杂棋局概念。我们同时借助SAEs和线性探针研究线性探针准确率与棋子颜色特征的演进过程，比较二者捕捉模型所学内容的有效性。尽管我们从较小的语言模型OthelloGPT展开研究，但本研究为更广泛地理解GPT模型、Transformer和LLMs学习到的内部表征建立了框架。我们的代码已公开：https://github.com/ALT-JS/OthelloSAE。
英文摘要：
Large Language Models (LLMs) excel at tasks like language processing, strategy games, and reasoning but struggle to build generalizable internal representations essential for adaptive decision-making in agents. For agents to effectively navigate complex environments, they must construct reliable world models. While LLMs perform well on specific benchmarks, they often fail to generalize, leading to brittle representations that limit their real-world effectiveness. Understanding how LLMs build internal world models is key to developing agents capable of consistent, adaptive behavior across tasks. We analyze OthelloGPT, a GPT-based model trained on Othello gameplay, as a controlled testbed for studying representation learning. Despite being trained solely on next-token prediction with random valid moves, OthelloGPT shows meaningful layer-wise progression in understanding board state and gameplay. Early layers capture static attributes like board edges, while deeper layers reflect dynamic tile changes. To interpret these representations, we compare Sparse Autoencoders (SAEs) with linear probes, finding that SAEs offer more robust, disentangled insights into compositional features, whereas linear probes mainly detect features useful for classification. We use SAEs to decode features related to tile color and tile stability, a previously unexamined feature that reflects complex gameplay concepts like board control and long-term planning. We study the progression of linear probe accuracy and tile color using both SAE's and linear probes to compare their effectiveness at capturing what the model is learning. Although we begin with a smaller language model, OthelloGPT, this study establishes a framework for understanding the internal representations learned by GPT models, transformers, and LLMs more broadly. Our code is publicly available: https://github.com/ALT-JS/OthelloSAE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
