arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.03735cs.CL
收藏

语言模型线性地编码数字的值

Language Models Encode the Value of Numbers Linearly

Peking University(北京大学)

Fangwei Zhu, Damai Dai, Zhifang Sui

AI总结:

本文通过线性探针和向量干预实验,揭示了大型语言模型在不同层上线性编码数字值并存储计算结果的机制,为理解和利用LLMs中的数值信息提供了证据。

摘要:

大型语言模型(LLMs)在各种任务中展现出令人印象深刻的能力,但其在数学问题上的内部机制仍待深入探索。本文研究了一个基本问题:语言模型如何编码数字(数学中的基本元素)的值。为研究此问题,我们构建了一个包含加法问题的合成数据集,并利用线性探针从隐藏状态中读取输入数字。实验结果证实了LLMs在不同层上存在编码的数字值,且这些值可通过线性探针提取。进一步实验表明,LLMs以类似方式存储计算结果,我们可以通过简单的向量加法干预输出,证明了编码数字与语言模型输出之间的因果联系。我们的研究为LLMs线性编码数字值提供了证据,为更好地探索、设计和利用LLMs中的数值信息提供了见解。

英文摘要:

Large language models (LLMs) have exhibited impressive competence in various tasks, but their internal mechanisms on mathematical problems are still under-explored. In this paper, we study a fundamental question: how language models encode the value of numbers, a basic element in math. To study the question, we construct a synthetic dataset comprising addition problems and utilize linear probes to read out input numbers from the hidden states. Experimental results support the existence of encoded number values in LLMs on different layers, and these values can be extracted via linear probes. Further experiments show that LLMs store their calculation results in a similar manner, and we can intervene the output via simple vector additions, proving the causal connection between encoded numbers and language model outputs. Our research provides evidence that LLMs encode the value of numbers linearly, offering insights for better exploring, designing, and utilizing numeric information in LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑