arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.04520cs.CL
收藏

语言模型组合泛化与幻觉中的线性相关性

Linear Correlation in LM's Compositional Generalization and Hallucination

University of California, San Diego(加利福尼亚大学圣迭戈分校)

Letian Peng, Chenyang An, Shibo Hao, Chengyu Dong, Jingbo Shang

AI总结:

本文发现语言模型在知识组合中存在线性相关现象,该线性变换对微调具有鲁棒性,既能泛化更新知识也会导致幻觉,并证明其可借助预训练词汇表示学习,可作为模型泛化能力的识别指标。

摘要:

语言模型(LMs)的泛化能力正引发激烈争论,一方面是其具备通用智能的潜力,另一方面则是其在基础知识组合上表现困难(例如反向/转换诅咒)。本文揭示了语言模型在知识组合过程中存在的线性相关现象。具体而言,在某些相关知识之间存在一种线性变换,可以将一个提示的下一个 token 预测 logits 映射到另一个提示,例如对于任意给定的 X,从“X 居住在城市”映射到“X 居住在国家”。这反映了人类知识组合中的线性特征,例如巴黎→法国。我们的研究结果表明,该线性变换对大规模微调具有鲁棒性:当与实际世界关系一致时,它能够泛化更新后的知识;但当偏离实际关系时,则会导致幻觉。实验结果表明,线性相关性可以作为语言模型泛化能力的潜在识别指标。最后,我们证明这种线性相关性可以通过单个前馈网络和预训练词汇表示来学习,这表明语言模型的泛化在很大程度上依赖于后者。

英文摘要:

The generalization of language models (LMs) is undergoing active debates, contrasting their potential for general intelligence with their struggles with basic knowledge composition (e.g., reverse/transition curse). This paper uncovers the phenomenon of linear correlations in LMs during knowledge composition. For explanation, there exists a linear transformation between certain related knowledge that maps the next token prediction logits from one prompt to another, e.g., "X lives in the city of" $\rightarrow$ "X lives in the country of" for every given X. This mirrors the linearity in human knowledge composition, such as Paris $\rightarrow$ France. Our findings indicate that the linear transformation is resilient to large-scale fine-tuning, generalizing updated knowledge when aligned with real-world relationships, but causing hallucinations when it deviates. Empirical results suggest that linear correlation can serve as a potential identifier of LM's generalization. Finally, we show such linear correlations can be learned with a single feedforward network and pre-trained vocabulary representations, indicating LM generalization heavily relies on the latter.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑