Audit-of-Understanding：面向语言模型数学推理的后验约束推理
Audit-of-Understanding: Posterior-Constrained Inference for Mathematical Reasoning in Language Models
AI总结：
提出 AoU 后验约束推理框架，通过分解、审计并仅采用已验证假设来抑制数学推理幻觉，并给出理论保证、风险界与显著基准提升。
摘要：
大型语言模型（LLM）通常会生成看似连贯、但建立在无支撑假设之上的推理轨迹，从而导致幻觉式结论。以往工作主要处理事实性幻觉，或依赖事后验证，导致由推理引发的幻觉在很大程度上仍未得到解决。我们提出 Audit-of-Understanding（AoU）框架，通过三个阶段将推理约束在经过验证的前提上：（1）将查询分解为候选假设，（2）审计这些假设的支撑依据，（3）仅以经过验证的子集为条件进行推理。形式上，AoU 是后验约束推理，与选择性预测和拒绝学习相关联。我们的贡献有三方面：（i）完美验证下的理论保证，（ii）不完美审计下的超额风险界，以及（iii）可处理性分析。实验上，AoU 在 GSM8K、MultiArith 和 SVAMP 上同时提升了准确率与忠实度；相较于 Chain-of-Thought、Self-Consistency 和 CoT-Decoding，其在 GSM8K 上最高取得 +30% 的增益，在 MultiArith 上取得 +45%，并在 SVAMP 上取得稳定的 +20–28% 提升。代码位于 https://anonymous.4open.science/r/audit-of-understanding-E28B。
英文摘要：
Large language models (LLMs) often generate reasoning traces that appear coherent but rest on unsupported assumptions, leading to hallucinated conclusions. Prior work mainly addresses factual hallucinations or relies on post-hoc verification, leaving reasoning-induced hallucinations largely unaddressed. We propose Audit-of-Understanding (AoU), a framework that constrains inference to validated premises through three phases: (1) decomposing a query into candidate assumptions, (2) auditing their support, and (3) conditioning inference only on the validated subset. Formally, AoU is \emph{posterior-constrained inference}, connecting to selective prediction and rejection learning. Our contributions are threefold: (i) theoretical guarantees under perfect validation, (ii) excess-risk bounds under imperfect audits, and (iii) tractability analysis. Empirically, AoU improves both accuracy and faithfulness on GSM8K, MultiArith, and SVAMP, achieving up to +30% gains on GSM8K, +45% on MultiArith, and consistent +20--28% improvements on SVAMP over Chain-of-Thought, Self-Consistency, and CoT-Decoding. Code is available at https://anonymous.4open.science/r/audit-of-understanding-E28B.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
