arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2507.17618cs.CLcs.PF
收藏

SimLens用于大型语言模型中的早期退出:通过一个额外的标记获取准确的潜在预测

SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Token

Institute of Neuroscience, State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences(中国科学院脑科学与智能技术卓越创新中心神经科学研究所,脑认知与脑启发智能技术国家重点实验室,脑科学与智能技术卓越创新中心);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院)

Ming Ma, Bowen Zheng, Zhongqiao Lin, Tianming Yang

AI总结:

本文提出SimLens,一种无需训练的解码器,通过保留起始标记和候选答案标记进行轻量级延续,提升潜在预测准确性。结合线性SimLens和SimExit机制,在多个数据集上实现更高的准确性和更快的推理速度。

摘要:

大型语言模型(LLMs)中间层的预测信息丰富但难以准确解码,尤其在早期层。现有镜头式方法通常依赖直接线性读出,简单但容易偏离模型最终预测。我们提出SimLens,一种简单的训练自由解码器,仅保留起始标记和候选答案标记([s]和[a]),并通过剩余上层进行一次轻量级延续。这种意外的小修改恢复了比直接线性解码更准确的潜在预测。我们进一步引入线性SimLens,一种轻量线性近似用于基于熵的置信度估计,并在SimExit中结合两者,形成混合早期退出机制。在ARC、BoolQ和HeadQA数据集上,使用LLaMA-7B和Vicuna-7B,SimLens在所有六个设置中提高了Iso-Compute准确性,即使公平计算包括额外的两个标记后置开销,平均增益为+0.43。SimExit在最佳准确性操作点上平均提速1.15倍,在允许最多1个百分点准确性下降时提速1.40倍。消融实验显示,[s]和[a]分别扮演全局条件和语义锚点的不同角色。

英文摘要:

Intermediate-layer predictions in large language models (LLMs) are informative but hard to decode accurately, especially at early layers. Existing lens-style methods typically rely on direct linear readout, which is simple but often drifts away from the model's eventual prediction. We proposeSimLens, a simple training-free decoder for single-token decision tasks that keeps only the start token and a candidate answer token ([s] and [a]) and performs one lightweight continuation through the remaining upper layers. This surprisingly small modification recovers much more accurate latent predictions than direct linear decoding. We further introduce Linear SimLens, a lightweight linear approximation for entropy-based confidence estimation, and combine the two in SimExit, a hybrid early-exit mechanism. On ARC, BoolQ, and HeadQA with LLaMA-7B and Vicuna-7B, SimLens improves Iso-Compute accuracy in all six settings, with an average gain of +0.43 even when fair compute includes the extra two-token post-forward overhead. SimExit yields an average 1.15$\times$ speedup at the best-accuracy operating points and 1.40$\times$ when allowing up to a 1 percentage-point accuracy drop. Ablations show that [s] and [a] play distinct roles as global condition and semantic anchor, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑