大语言模型与扩展丘奇-图灵论题
Large Language Models and the Extended Church-Turing Thesis
AI总结：
本研究从可计算性理论角度探究大语言模型的计算能力，证明固定LLM等价于确定性有限状态转换器，演化的LLM谱系等价于带外部信息的交互图灵机，验证了扩展丘奇-图灵论题对LLM谱系的有效性，且其具备超图灵计算能力。
摘要：
扩展丘奇-图灵论题（ECTT）假定，所有有效信息处理过程，包括无界且非一致的交互计算，都可以用带外部信息的交互图灵机来描述。这一论断是否也适用于当代大语言模型（LLM）的能力？从更宏观的视角来看，这个问题需要借助可计算性与计算复杂性理论的经典方法，尤其是自动机理论，来研究大语言模型的计算能力。\n围绕这一方向，我们取得了若干基础性成果。首先，我们论证了任意固定（非自适应）的大语言模型，其计算能力等价于一个规模可能极大的确定性有限状态转换器，这一结论刻画了大语言模型的基础计算层级。我们将这一结论拓展，得到了关于大语言模型模拟空间有界图灵机的核心结果。其次，我们证明了不断演化的大语言模型谱系，其计算能力等价于带外部信息的交互图灵机。后一项发现证实了扩展丘奇-图灵论题对大语言模型谱系的有效性；从可计算性角度来看，这也意味着大语言模型谱系具备超图灵计算能力。因此，在我们的计算模型中，知识生成总体上是由大语言模型谱系实现的非算法过程。最后，我们在多个相关学科与哲学的更广泛背景下，讨论了本研究发现的价值。
英文摘要：
The Extended Church-Turing Thesis (ECTT) posits that all effective information processing, including unbounded and non-uniform interactive computations, can be described in terms of interactive Turing machines with advice. Does this assertion also apply to the abilities of contemporary large language models (LLMs)? From a broader perspective, this question calls for an investigation of the computational power of LLMs by the classical means of computability and computational complexity theory, especially the theory of automata. Along these lines, we establish a number of fundamental results. Firstly, we argue that any fixed (non-adaptive) LLM is computationally equivalent to a, possibly very large, deterministic finite-state transducer. This characterizes the base level of LLMs. We extend this to a key result concerning the simulation of space-bounded Turing machines by LLMs. Secondly, we show that lineages of evolving LLMs are computationally equivalent to interactive Turing machines with advice. The latter finding confirms the validity of the ECTT for lineages of LLMs. From a computability viewpoint, it also suggests that lineages of LLMs possess super-Turing computational power. Consequently, in our computational model knowledge generation is in general a non-algorithmic process realized by lineages of LLMs. Finally, we discuss the merits of our findings in the broader context of several related disciplines and philosophies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
