涌现的世界表征:探究在合成任务上训练的序列模型
Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
AI总结:
本研究通过让GPT变体预测奥赛罗棋步,发现网络涌现出非线性的棋盘状态内部表征,可控制输出并生成解释预测的潜在显著性图。
摘要:
语言模型展现出令人惊讶的广泛能力,但其表面能力的来源尚不清楚。这些网络仅仅是记忆了表面统计数据的集合,还是依赖于对生成其所见序列过程的内部表征?我们通过将GPT模型的一种变体应用于预测简单棋盘游戏奥赛罗中合法走子的任务来研究这一问题。尽管网络对游戏或其规则没有先验知识,我们发现了关于棋盘状态的一种涌现的非线性内部表征的证据。干预实验表明,这种表征可用于控制网络的输出,并创建“潜在显著性图”,从而以人类可理解的方式帮助解释预测。
英文摘要:
Language models show a surprising range of capabilities, but the source of their apparent competence is unclear. Do these networks just memorize a collection of surface statistics, or do they rely on internal representations of the process that generates the sequences they see? We investigate this question by applying a variant of the GPT model to the task of predicting legal moves in a simple board game, Othello. Although the network has no a priori knowledge of the game or its rules, we uncover evidence of an emergent nonlinear internal representation of the board state. Interventional experiments indicate this representation can be used to control the output of the network and create "latent saliency maps" that can help explain predictions in human terms.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。