发表机构
Arizona State University; University of Maryland; Morgan Stanley（亚利桑那州立大学; 马里兰大学; 摩根士丹利）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出理论框架分析大语言模型在上下文影响下的推理动态，发现重复断言不累积证据，预测表示收敛至稳定机制，从而揭示上下文说服力的内在极限。
AI 中文摘要
现代提示技术的核心在于上下文敏感性，即大语言模型根据推理时上下文调整其预测的能力。尽管这一能力至关重要，但在强上下文影响下的推理行为仍未被充分理解，尤其是在内部推理动态层面。我们引入了一个通过推理动态分析上下文影响的理论框架，使得能够在输出层面的答案变化之外，对推理行为进行定量刻画。我们的分析表明，在重复的上下文断言下，推理动态不会表现出无界的漂移。相反，预测表示会收敛到稳定的、依赖于查询的机制，这些机制从根本上限制了上下文信号能否改变模型的预测。这导致了一个令人惊讶的发现：重复的上下文断言在推理过程中并不作为累积证据，因此即使在无界重复的情况下也可能无法改变模型的预测，而在其他情况下，预测的改变则不可避免。我们实证验证了我们的理论预测，展示了理论与观察到的推理行为之间的强一致性。这些贡献为刻画推理过程中上下文影响的极限提供了一条原则性路径，并为模型开发提供了实际启示。
英文摘要
At the core of modern prompting techniques is contextual sensitivity, the ability of large language models to adapt their predictions based on inference-time context. Despite its central role, inference behavior under strong contextual influence remains poorly understood, particularly at the level of internal inference dynamics. We introduce a theoretical framework for analyzing contextual influence through inference dynamics, enabling quantitative characterization of inference behavior beyond output-level answer changes. Our analysis shows that inference dynamics do not exhibit unbounded drift under repeated contextual assertions. Instead, predictive representations converge to stable, query-dependent regimes that fundamentally constrain whether contextual signals can alter a model's prediction. This leads to a surprising finding: Repeated contextual assertions do not act as accumulating evidence during inference and may therefore fail to alter a model's prediction even under unbounded repetition, while in other cases a prediction change becomes inevitable. We empirically validate our theoretical predictions, demonstrating strong alignment between theory and observed inference behavior. These contributions offer a principled pathway toward characterizing the limits of contextual influence during inference, providing practical implications for model development.
CommentsPublished in the Proceedings of the 43rd International Conference on Machine Learning (ICML 2026)
Journal refProceedings of the 43rd International Conference on Machine Learning, PMLR 306:45465-45494, 2026
