大语言模型预测器知道但不说的内容：探究校准和忠实性的内部表示
What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness
AI总结：
研究针对预测微调的大语言模型校准和忠实性问题，通过在中间激活上训练表示池探测器，发现其校准效果更好，还能评估CoT忠实性、追踪行为变化等，证明探究内部表示可用于校准、审计和分类语言模型预测器及推理模型。
摘要：
针对预测进行微调的大语言模型可能准确但校准不佳，其思维链（CoT）推理可能无法如实反映预测背后的证据。我们研究内部表示是否能更直接地洞察这两者。在OpenForesight上使用Eternis - Forecaster 8B，我们在中间激活上训练表示池探测器，发现其校准效果显著更好，对GLM - 4.7 - Flash和GLM - 4.5 - Air也成立。通过证据消融和转移注入评估CoT忠实性，这些探测器还能追踪行为变化、预测变化方向。强制回答表明预测在推理开始前大多已确定，按此分布路由问题可节省生成令牌。这些结果确立了探究内部表示是校准、审计和分类语言模型预测器及推理模型的实用工具。
英文摘要：
Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both. Working with Eternis-Forecaster 8B on OpenForesight, we train representation-pooling probes on intermediate activations and find they achieve substantially better calibration; a result that also holds for GLM-4.7-Flash and GLM-4.5-Air. We then assess CoT faithfulness through evidence ablation and diversionary injection: removing an influential source in the prompt often changes the model's forecast while leaving the reasoning trace untouched. The same probes function as lie detectors: their activations track behavioral shifts far better than the reasoning trace does, and they also predict the direction of change in 84% of cases, including when the CoT conceals the perturbation's influence. Finally, forced answering reveals that forecasts are largely fixed before reasoning begins: a single pre-reasoning pass recovers the committed answer and confidence, and routing questions by the spread of this pre-set answer distribution saves 30-47% of generated tokens, with no loss of accuracy. Together, these results establish probing internal representations as a practical tool for calibrating, auditing, and triaging language model forecasters and reasoning models more broadly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
