arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.10816cs.CLcs.AI
收藏

超越置信度:生成模型推理的节奏

Beyond Confidence: The Rhythms of Reasoning in Generative Models

Harbin Institute of Technology(哈尔滨工业大学)

Deyuan Liu, Zecheng Wang, Zhanyue Qin, Zhiying Tu, Dianhui Chu, Dianbo Sui

AI总结:

本文提出δ_TC B度量标准,用于评估生成模型在预测稳定性方面的鲁棒性,揭示了传统度量在上下文学习中的不足,并为提升模型稳定性提供新方法。

摘要:

大型语言模型(LLMs)展现出令人印象深刻的能力,但对微小的输入上下文变化却极为敏感,影响了其可靠性。传统的度量标准如准确率和困惑度无法评估局部预测鲁棒性,因为标准化的输出概率可能掩盖了LLM内部状态对扰动的内在韧性。我们引入了令牌约束界(δ_TC B),一种新的度量标准,量化了LLM在主导下一个令牌预测显著变化之前能承受的最大内部状态扰动。本质上与输出嵌入空间几何学相关,δ_TC B提供了关于模型内部预测承诺稳定性的见解。我们的实验显示δ_TC B与有效的提示工程相关联,并揭示了在上下文学习和文本生成过程中被困惑度所遗漏的关键预测不稳定性。δ_TC B提供了一种原则性、互补的方法来分析和 potentially 改善LLM预测的上下文稳定性。

英文摘要:

Large Language Models (LLMs) exhibit impressive capabilities yet suffer from sensitivity to slight input context variations, hampering reliability. Conventional metrics like accuracy and perplexity fail to assess local prediction robustness, as normalized output probabilities can obscure the underlying resilience of an LLM's internal state to perturbations. We introduce the Token Constraint Bound ($δ_{\mathrm{TCB}}$), a novel metric that quantifies the maximum internal state perturbation an LLM can withstand before its dominant next-token prediction significantly changes. Intrinsically linked to output embedding space geometry, $δ_{\mathrm{TCB}}$ provides insights into the stability of the model's internal predictive commitment. Our experiments show $δ_{\mathrm{TCB}}$ correlates with effective prompt engineering and uncovers critical prediction instabilities missed by perplexity during in-context learning and text generation. $δ_{\mathrm{TCB}}$ offers a principled, complementary approach to analyze and potentially improve the contextual stability of LLM predictions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑