信任，但不验证：LLM 源评估中的认知盲点
Trust, but Don't Verify: Epistemic Blind Spots in LLM Source Evaluation
AI总结：
研究语言模型在多源综合中是否评估证据质量，发现模型虽能检测伪造统计但未在综合中启用，而是依赖方法论-语域门控，导致数值有效性被抑制。
摘要：
语言模型日益充当认知代理，综合多个来源的证据以辅助决策。然而，它们是否评估这些证据的质量，还是仅仅基于表面呈现进行聚合，目前尚不清楚。我们表明，模型具备检测伪造统计数据的能力（孤立方法论的正确识别率为0.76-1.00），但在多源综合过程中并未启用这一能力，无论统计数据是伪造还是有效，都会产生相似的数值估计。具体而言，源影响受方法论-语域门控支配，该门控响应分析文本的分布性语域，但不响应数值有效性：例如，统计上不可能的置信区间与有效区间获得相同权重。这种行为分离在来自三个家族（Claude、Qwen、OLMo）的五个模型以及三个专业领域中均得到复现。机制分析（包括因果追踪、线性探针和组件级归因）收敛于同一解释：模型编码并因果使用一种跨领域转移的方法论-语域表示（探针AUC 0.83-0.92），而数值有效性信号（在孤立时可解码）在多源综合中被抑制至随机水平。基于提示的缓解措施（甚至是指定精确统计检查的预言清单）会产生全面怀疑而非选择性辨别，我们检查的后训练流程强化了风格捷径而未建立数值验证。与追踪用户偏好的奉承行为不同，这种失败追踪的是源是否呈现为分析可信，而非其主张是否内部一致。我们称之为认知对齐：与偏好对齐和安全对齐一样，问题不在于能力，而在于部署。
英文摘要：
Language models increasingly act as epistemic proxies, synthesizing evidence from multiple sources to inform decisions. Whether they evaluate the quality of that evidence, or merely aggregate it based on surface presentation, remains poorly understood. We show that models possess the capability to detect fabricated statistics in isolation but do not recruit this capability during multi-source synthesis, producing similar numeric estimates whether the statistics are fabricated or valid. Specifically, source influence is governed by a methodology-register gate that responds to the distributional register of analytical text but not to numeric validity: for example, statistically impossible confidence intervals receive the same weight as valid ones. The behavioral dissociation replicates across six models from four families (Anthropic Claude, Qwen, OLMo, and OpenAI GPT-5.4) and three professional domains. Mechanistic analyses, including causal tracing, linear probes, and component-level attribution, converge on the same account: the model encodes and causally uses a methodology-register representation that transfers across domains, while numeric-validity signals, decodable in isolation, are suppressed to chance during multi-source synthesis. Prompting-based mitigations, even an oracle checklist naming the exact statistical checks, produce blanket skepticism rather than selective discernment, and the post-training pipelines we examine reinforce the shortcut without building numeric verification. Unlike sycophancy, which tracks user preference, this failure tracks whether a source presents as analytically credible, not whether its claims are consistent. We term this \textit{epistemic alignment}: like preference and safety alignment, the question is not capability but deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
