arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.15054cs.CLcs.AIcs.LG
收藏

揭示语言模型中的 Token 预测精炼过程并识别关键层

Unraveling Token Prediction Refinement and Identifying Essential Layers in Language Models

Independent Researcher(独立研究者)

Jaturong Kongmanee

AI总结:

该研究利用 logit lens 分析 LLM 中间层的 token 预测,揭示相关信息位置对精炼深度的倒 U 形影响,并指出不同层对最终正确输出的重要性并不相同。

摘要:

本研究旨在揭示大型语言模型(LLM)如何通过内部处理迭代地精炼 token 预测。我们采用 logit lens 技术来分析模型由中间表示得到的 token 预测。具体而言,我们关注:(1)LLM 如何访问并利用来自输入上下文的信息;(2)相关信息的位置如何影响模型的 token 预测精炼过程。在一项输入上下文长度变化的多文档问答任务中,我们发现,预测精炼深度(定义为 LLM 从最初的正确 token 预测过渡到最终、稳定的正确输出所使用的中间层数)作为相关信息位置的函数,呈现近似倒 U 形曲线。我们还发现,当相关信息位于输入上下文的开头或结尾时,这两个层之间的差距平均会缩小。这表明,当处理相关信息位于中间的较长上下文时,模型需要更多次精炼。此外,我们的研究结果表明,并非所有层对于确定最终正确输出都同等关键。我们的分析为 token 预测在不同条件下如何分布提供了洞见,并与 AI 安全研发中已有的假设和先前发现建立了重要联系。

英文摘要:

This research aims to unravel how large language models (LLMs) iteratively refine token predictions through internal processing. We utilized a logit lens technique to analyze the model's token predictions derived from intermediate representations. Specifically, we focused on (1) how LLMs access and utilize information from input contexts, and (2) how positioning of relevant information affects the model's token prediction refinement process. On a multi-document question answering task with varying input context lengths, we found that the depth of prediction refinement (defined as the number of intermediate layers an LLM uses to transition from an initial correct token prediction to its final, stable correct output), as a function of the position of relevant information, exhibits an approximately inverted U-shaped curve. We also found that the gap between these two layers, on average, diminishes when relevant information is positioned at the beginning or end of the input context. This suggested that the model requires more refinements when processing longer contexts with relevant information situated in the middle. Furthermore, our findings indicate that not all layers are equally essential for determining final correct outputs. Our analysis provides insights into how token predictions are distributed across different conditions, and establishes important connections to existing hypotheses and previous findings in AI safety research and development.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑