Transformer 需要眼镜!语言任务中的信息过度挤压
Transformers need glasses! Information over-squashing in language tasks
AI总结:
本文通过理论信号传播分析证明 decoder-only Transformer 存在表示坍缩与信息过度挤压,并在当代 LLM 上实证验证及提出简单缓解思路。
摘要:
我们研究信息在 decoder-only Transformer 中的传播方式,这类架构是大多数现有前沿大型语言模型(LLM)的主干。我们采用理论信号传播分析,具体而言,分析 Transformer 最后一层中最后一个 token 的表示,因为该表示用于下一 token 预测。分析揭示了一种表示坍缩现象:我们证明,输入 Transformer 的某些不同序列会在最后一个 token 处产生任意接近的表示。现代 LLM 中经常使用的低精度浮点格式会加剧这一效应。因此,可证明模型无法以不同方式响应这些序列,从而在计数或复制等任务中导致错误。此外,我们表明 decoder-only Transformer 语言模型可能丧失对输入中特定 token 的敏感性,这与图神经网络中著名的过度挤压现象有关。我们在当代 LLM 上提供了支持这些论断的实证证据。我们的理论还指出了缓解这些问题的简单解决方案。
英文摘要:
We study how information propagates in decoder-only Transformers, which are the architectural backbone of most existing frontier large language models (LLMs). We rely on a theoretical signal propagation analysis -- specifically, we analyse the representations of the last token in the final layer of the Transformer, as this is the representation used for next-token prediction. Our analysis reveals a representational collapse phenomenon: we prove that certain distinct sequences of inputs to the Transformer can yield arbitrarily close representations in the final token. This effect is exacerbated by the low-precision floating-point formats frequently used in modern LLMs. As a result, the model is provably unable to respond to these sequences in different ways -- leading to errors in, e.g., tasks involving counting or copying. Further, we show that decoder-only Transformer language models can lose sensitivity to specific tokens in the input, which relates to the well-known phenomenon of over-squashing in graph neural networks. We provide empirical evidence supporting our claims on contemporary LLMs. Our theory also points to simple solutions towards ameliorating these issues.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。