arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.03415cs.CLcs.AI
收藏

越远离分布,表示越稀疏:分析LLM中的OOD机制

Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs

Rutgers University(罗格斯大学);Northwestern University(西北大学);UKP Lab, TU Darmstadt(德累斯顿技术大学UKP实验室);New Jersey Institute of Technology(新泽西理工学院);NEC Lab American(美国NEC实验室);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)

Mingyu Jin, Yutong Yin, Jingcheng Niu, Qingcheng Zeng, Wujiang Xu, Mengnan Du, Wei Cheng, Zhaoran Wang, Tianlong Chen, Dimitris N. Metaxas

AI总结:

研究LLM在面对难度增加的输入时内部表示的适应机制,发现任务难度提升导致最后隐藏层表示稀疏化,提出基于稀疏性的课程学习策略提升性能。

摘要:

本文研究了大型语言模型(LLM)在遇到难度增加的输入时如何调整其内部表示,通过量化分布外(OOD)偏移程度发现,随着任务难度增加,LLM的最后隐藏层表示显著稀疏化。通过控制分析和学习动态解释,证明这种稀疏性是稳定推理的适应机制,并设计了基于稀疏性的课程学习策略,显著提升了性能。研究提供了关于LLM如何内部化OOD挑战的新机制性见解。代码库可访问:https://github.com/MingyuJ666/sparsityLLM。

英文摘要:

In this work, we investigate how the internal representations of Large Language Models (LLMs) change with inputs of increasing difficulty. Although sparsity changes may not be obvious for individual samples, dataset-level results reveal a consistent statistical trend: as task difficulty increases (e.g., harder questions, longer contexts, or more answer choices), the last hidden states of LLMs become systematically sparser. In short, \textbf{the harder the task, the sparser the representation}. This sparsity--difficulty relation is observable across diverse models and domains, suggesting that harder inputs drive more concentrated activation patterns in the last hidden state. Through a series of controlled analyses and learning-dynamics experiments, we show that representational density is a learned property of data familiarity: models develop rich, distributed representations for mastered patterns, while unfamiliar inputs default to sparser activations. Our finding is also actionable. We illustrate this with Sparsity-Guided Curriculum In-Context Learning (SG-ICL), which uses sparsity to select few-shot demonstrations matched to the query's difficulty, outperforming standard CoT and Auto-CoT baselines on MATH-500. Our study provides new insights into how LLM representations reflect task difficulty and how this signal can be leveraged for inference. The source code is available at the URL: https://github.com/MingyuJ666/sparsityLLM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑