arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.11206cs.CL
收藏

指令塑造语言的生成,而非处理

Instructions Shape Production of Language, not Processing

Department of Linguistics, University of Tübingen(图宾根大学语言学系);IBM Research, MIT, and MIT-IBM Watson AI Lab(IBM研究院、麻省理工学院和麻省理工-IBM沃森人工智能实验室);Interdisciplinary Transformation University of Austria(奥地利跨学科转型大学);IBM Research(IBM研究院)

Andreas Waldis, Leshem Choshen, Yufang Hou, Yotam Perlitz

AI总结:

研究揭示指令通过影响语言模型生成阶段的信息 asymmetry,而非处理阶段,通过五种二元判断任务层析任务特定信息,发现输出 tokens 的信息变化与行为强相关,而输入 tokens 的信息变化弱相关。

摘要:

指令触发语言模型中的生成中心机制。通过以认知启发式视角分离语言处理与生成,我们通过五种二元判断任务,层析任务特定信息,揭示这种机制为两个阶段之间的不对称性。具体而言,我们测量指令 tokens 如何在样本 tokens 被处理和输出 tokens 被生成时塑造信息。在不同提示变体中,样本 tokens 中的任务特定信息保持稳定且仅弱相关于行为,而输出 tokens 中的相同信息则显著变化且强相关于行为。基于注意力的干预证实了这种模式的因果性:阻断指令流到所有后续 tokens 会减少输出 tokens 的行为和信息,而仅阻断到样本 tokens 则对两者影响很小。这种不对称性在不同模型家族和任务中普遍适用,并随着模型规模和指令微调而变得更加明显。我们的发现表明,理解模型能力需要联合评估内部和行为,同时通过 token 位置分解内部视角以区分输入 tokens 的处理与输出 tokens 的生成。

英文摘要:

Instructions trigger a production-centered mechanism in language models. Through a cognitively inspired lens that separates language processing and production, we reveal this mechanism as an asymmetry between the two stages by probing task-specific information layer-wise across five binary judgment tasks. Specifically, we measure how instruction tokens shape information both when sample tokens, the input under evaluation, are processed and when output tokens are produced. Across prompting variations, task-specific information in sample tokens remains largely stable and correlates only weakly with behavior, whereas the same information in output tokens varies substantially and correlates strongly with behavior. Attention-based interventions confirm this pattern causally: blocking instruction flow to all subsequent tokens reduces both behavior and information in output tokens, whereas blocking it only to sample tokens has minimal effect on either. The asymmetry generalizes across model families and tasks, and becomes sharper with model scale and instruction-tuning, both of which disproportionately affect the production stage. Our findings suggest that understanding model capabilities requires jointly assessing internals and behavior, while decomposing the internal perspective by token position to distinguish the processing of input tokens from the production of output tokens.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑