当小模型适用于错误的原因：信任代理的过程验证
When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agents
AI总结：
本文揭示小型语言模型在推理过程中的可靠性问题，提出推理完整性评分，并展示RAG提升推理质量而元认知可能损害性能，强调过程验证对可信代理的重要性。
摘要：
部署小型语言模型（7-9B参数）作为自主代理需要对其推理过程建立信任，而非仅仅依赖输出结果。我们揭示了一个关键的可靠性危机：这些模型正确答案中50-69%的推理存在根本性缺陷——一种“正确但原因错误”的现象，这种现象无法被标准准确性指标所察觉。通过对10734个推理轨迹进行分析，涵盖三个模型和多样化的任务，我们引入了推理完整性分数（RIS），一种基于过程的度量标准，其通过显著的评分者一致性（κ=0.657）得到了验证。我们的发现挑战了传统做法：虽然检索增强生成（RAG）显著提高了推理完整性（Cohen's d=0.23-0.93），但元认知干预如自我批评在小型模型上往往损害性能（d=-0.14到-0.33）。机制分析显示RAG通过将计算基础置于外部证据上，使错误率减少了7.6%，而元认知则放大了困惑，缺乏足够的模型容量。为了实现部署，验证能力被提炼成一个神经分类器，其在100倍速度提升下实现了0.86的F1分数。这些结果强调了基于过程的验证对于可信代理的必要性：仅凭准确性是危险的，当模型可以因完全错误的原因而正确。
英文摘要：
Deploying small language models (7-9B parameters) as autonomous agents requires trust in their reasoning, not just their outputs. We reveal a critical reliability crisis: 50-69\% of correct answers from these models contain fundamentally flawed reasoning -- a ``Right-for-Wrong-Reasons'' phenomenon invisible to standard accuracy metrics. Through analysis of 10,734 reasoning traces across three models and diverse tasks, we introduce the Reasoning Integrity Score (RIS), a process-based metric validated with substantial inter-rater agreement ($κ=0.657$). Conventional practices are challenged by our findings: while retrieval-augmented generation (RAG) significantly improves reasoning integrity (Cohen's $d=0.23$--$0.93$), meta-cognitive interventions like self-critique often harm performance ($d=-0.14$ to $-0.33$) in small models on the evaluated tasks. Mechanistic analysis reveals RAG succeeds by grounding calculations in external evidence, reducing errors by 7.6\%, while meta-cognition amplifies confusion without sufficient model capacity. To enable deployment, verification capabilities are distilled into a neural classifier achieving 0.86 F1-score with 100$\times$ speedup. These results underscore the necessity of process-based verification for trustworthy agents: accuracy alone is dangerously insufficient when models can be right for entirely wrong reasons.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
