发表机构
Yale University; Johns Hopkins University; New York University; Microsoft Research（耶鲁大学; 约翰斯·霍普金斯大学; 纽约大学; 微软研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究发现神经网络内部表征隐含符号结构，可通过符号结构近似其向量表征，还能精准干预修改大型语言模型行为，为调和智能的符号概念与现代AI的向量本质提供了新途径。
AI 中文摘要
现代人工智能（AI）系统在看似适配性不佳的领域表现出色。传统上，智能被建模为在符号的结构化组合（如逻辑公式）上运行。然而，最强的现代AI系统基于神经网络，神经网络却用连续向量表示信息。向量似乎不足以捕捉语言、逻辑及其他认知领域的结构，但神经网络在这些领域取得了出色的性能。它们是如何做到的？在本研究中，我们提出一个潜在答案：尽管表面如此，神经网络的内部表征可能隐含地实现了符号结构。为支持该假设，我们表明多种神经网络的向量表征可通过符号结构紧密近似：我们可用实例化符号结构的闭式方程替换网络的整个表征生成过程，网络行为仍基本不变。这一发现既适用于训练以处理列表的小规模神经网络，也适用于在符号传统核心的四个领域（算术、逻辑、计算机代码和语言）运行的大型语言模型（LLMs）。此外，我们的符号近似允许通过对LLM内部表征的精准干预，以定向方式修改其行为，表明LLM的行为依赖于我们识别出的符号结构。本研究为调和长期存在的智能符号概念与现代AI的基于向量的本质提供了潜在途径。
英文摘要
Modern systems in artificial intelligence (AI) somehow excel in domains for which they seem poorly suited. Intelligence has traditionally been modeled as operating over structured combinations of symbols, such as logical formulas. However, the strongest modern AI systems are based on neural networks, which instead represent information in continuous vectors. Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas. How do they do it? In this work, we propose a potential answer: Despite appearances, perhaps the internal representations of neural networks implicitly realize symbolic structure. In support of this hypothesis, we show that the vector representations of a variety of neural networks can be closely approximated with symbolic structures: we can replace the network's entire representation-generating process with a closed-form equation instantiating a symbolic structure, and the network's behavior remains largely unchanged. This finding holds for both small-scale neural networks trained to manipulate lists as well as large language models (LLMs) operating in four domains that are central in symbolic traditions: arithmetic, logic, computer code, and language. Further, our symbolic approximation allows us to modify an LLM's behavior in targeted ways via precise interventions on its internal representations, showing that the LLM's behavior is reliant on the symbolic structures we have identified. This work provides a potential way to reconcile longstanding symbolic conceptions of intelligence with the vector-based nature of modern AI.
Comments30 pages, plus 29 pages of references and appendices
