arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01239cs.CLcs.AI
收藏

大语言模型中的自我锚定校准漂移:多轮对话如何重塑模型信心

Self-Anchoring Calibration Drift in Large Language Models: How Multi-Turn Conversations Reshape Model Confidence

Independent Researcher(独立研究者)

Harshavardhan

AI总结:

研究发现大语言模型在多轮对话中出现自我锚定校准漂移,导致模型信心变化及校准误差波动。

摘要:

我们引入了自我锚定校准漂移(SACD),这是一种假设的大语言模型(LLMs)在多轮对话中逐步基于自身先前输出构建时,表现出表达信心系统性变化的倾向。我们报告了一项实证研究,比较了三个前沿模型——Claude Sonnet 4.6、Gemini 3.1 Pro和GPT-5.2——在涵盖事实性、技术性和开放性领域的150个问题上的表现,使用三种条件:单轮基线(A)、多轮自我锚定(B)和独立重复控制(C)。结果揭示了一种复杂且模型异质性的模式,部分偏离了预先注册的假设。Claude Sonnet 4.6在自我锚定下表现出显著下降的信心(均值CDS=-0.032,t(14)=-2.43,p=0.029,d=-0.627),同时显示出显著的校准误差漂移(F(4,56)=22.77,p<0.001,eta²=0.791)。GPT-5.2在开放性领域表现出相反的模式(均值CDS=+0.026),在第五轮时显著出现ECE上升。Gemini 3.1 Pro没有显著的CDS(t(14)=0.38,p=0.710),但其条件C数据揭示出一种引人注目的ECE模式:在没有自我锚定的情况下,Gemini的校准误差从0.327降至几乎零,而在自我锚定的情况下,ECE保持在约0.333——这表明SACD可以表现为对自然校准改进的抑制,而不是增强

英文摘要:

We introduce Self-Anchoring Calibration Drift (SACD), a hypothesized tendency for large language models (LLMs) to show systematic changes in expressed confidence when building iteratively on their own prior outputs across multi-turn conversations. We report an empirical study comparing three frontier models -- Claude Sonnet 4.6, Gemini 3.1 Pro, and GPT-5.2 -- across 150 questions spanning factual, technical, and open-ended domains, using three conditions: single-turn baseline (A), multi-turn self-anchoring (B), and independent repetition control (C). Results reveal a complex, model-heterogeneous pattern that partially diverges from pre-registered hypotheses. Claude Sonnet 4.6 exhibited significant decreasing confidence under self-anchoring (mean CDS = -0.032, t(14) = -2.43, p = .029, d = -0.627), while also showing significant calibration error drift (F(4,56) = 22.77, p < .001, eta^2 = .791). GPT-5.2 showed the opposite pattern in open-ended domains (mean CDS = +0.026) with significant ECE escalation by Turn 5. Gemini 3.1 Pro showed no significant CDS (t(14) = 0.38, p = .710), but its Condition C data reveals a striking ECE pattern: without self-anchoring, Gemini's calibration error drops from .327 to near zero across repetitions, whereas self-anchoring holds ECE flat at approximately .333 -- indicating that SACD can manifest as suppression of natural calibration improvement rather than ac

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑