问即得之：论提示词（prompting）的图灵完备性
Ask, and it shall be given: On the Turing completeness of prompting
AI总结：
本文首次从理论层面研究LLM提示词范式，证明有限规模Transformer可通过对应提示词计算任意可计算函数（即提示词具备图灵完备性），且能达到接近无界规模Transformer的复杂度界，为提示工程提供理论支撑。
摘要：
自GPT取得成功以来，大语言模型（LLM）正在革新机器学习领域，并催生了所谓的LLM提示词范式。在LLM时代，人们训练单个通用LLM，通过为其提供不同的提示词来执行不同任务。然而，这类实证成功在很大程度上缺乏理论支撑。据我们所知，本文首次针对LLM提示词范式开展理论研究。本研究证明，提示词实际上具备图灵完备性：存在一个有限规模的Transformer，使得对于任意可计算函数，都存在对应的提示词，让该Transformer能够计算此函数。此外我们还证明，即便仅使用单个有限规模的Transformer，它仍能达到与所有无界规模Transformer类几乎相同的复杂度界。总体而言，我们的研究结果揭示，提示词能够让单个有限规模Transformer具备高效通用性，为实际中的提示工程（prompt engineering）奠定了理论基础。
英文摘要：
Since the success of GPT, large language models (LLMs) have been revolutionizing machine learning and have initiated the so-called LLM prompting paradigm. In the era of LLMs, people train a single general-purpose LLM and provide the LLM with different prompts to perform different tasks. However, such empirical success largely lacks theoretical understanding. Here, we present the first theoretical study on the LLM prompting paradigm to the best of our knowledge. In this work, we show that prompting is in fact Turing-complete: there exists a finite-size Transformer such that for any computable function, there exists a corresponding prompt following which the Transformer computes the function. Furthermore, we show that even though we use only a single finite-size Transformer, it can still achieve nearly the same complexity bounds as that of the class of all unbounded-size Transformers. Overall, our result reveals that prompting can enable a single finite-size Transformer to be efficiently universal, which establishes a theoretical underpinning for prompt engineering in practice.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
