论Transformer与智能体的图灵完备性
On the Turing Completeness of Transformers and Agents
- SKLMS, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
- University of Chinese Academy of Sciences（中国科学院大学）
- Institute of AI for Industries, Chinese Academy of Sciences（中国科学院工业人工智能研究院）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
本文证明单个Transformer在无界输入下不具备图灵完备性，而由决策、执行和记忆模块组成的智能体架构可模拟任意图灵机，从而具备图灵完备性。
中文摘要 AI 辅助
Transformer已成为序列建模中的主导架构，在自然语言处理和推理任务中取得了显著成功。尽管现有文献已确立了Transformer在输入长度有界情况下的图灵完备性，但单个Transformer在处理无界长度输入时的推理能力尚未被充分探索。本文从理论上研究了单个Transformer的推理局限性以及智能体系统的增强能力。我们证明，单个固定的有限精度Transformer无法记忆某些输入长度任意的图灵机，例如算术运算；而单个固定的无限精度Transformer若以随机算法训练，在合理条件下以概率1不具备图灵完备性。为克服单个Transformer的局限性，我们定义了一种由决策、执行和记忆模块组成的正式智能体架构，并证明对于任意图灵机$\mathbb{T}$，存在一个智能体能够记忆$\mathbb{T}$，且其计算能力与$\mathbb{T}$等价。因此，智能体是图灵完备的。
英文摘要
Transformers have emerged as the dominant architecture in sequence modeling, achieving remarkable success in natural language processing and reasoning tasks. While existing literature has established the Turing completeness of transformers under bounded input length, the reasoning power of a single transformer operating on inputs of unbounded length is not fully explored. In this paper, we theoretically investigate the reasoning limitations of a single transformer and the enhanced capabilities of agent systems. We show that a single fixed finite precision transformer cannot memorize certain Turing machines with inputs of arbitrary length, such as the arithmetic; and a single fixed infinite precision transformer trained with a random algorithm is not Turing complete with probability one under reasonable conditions. To overcome the limitation of a single transformer, we define a formal agent architecture consisting of decision, execution, and memory modules and show that for any Turing machine $\mathbb{T}$, there exists an agent that can memorize $\mathbb{T}$ and is computationally the same as $\mathbb{T}$. Thus, agents are Turing complete.
