无金标准推理:自动形式化的代理-裁判理论
Reasoning without Gold Standards: A Proxy-Judge Theory of Autoformalization
AI总结:
提出无参考的代理-裁判框架,通过多轴属性检查替代金标准匹配,实现自动形式化的迭代优化,理论保证收敛,实验提升通过率。
摘要:
复杂的推理任务日益要求系统生成其正确性无法通过与单一参考精确匹配来判断的输出。自动形式化(AF)是一个代表性例子;它要求模型将非正式的数学或逻辑推理翻译成可形式化检查的对象,然而专家验证的形式化在玩具案例之外无法扩展,且一个非正式论证可以有许多有效的形式化呈现。因此,进展取决于部分、结构化的代理能否替代精确参考。我们为AF引入了一个无参考的代理-裁判框架,用多轴属性检查向量替代金标准匹配。该框架沿三个结构范围组织代理:涵盖所引发对象的全局属性、子组件内部的模块属性、以及将其重新对齐到非正式来源的跨域属性,并将每个轴聚合成一个裁决向量。该向量驱动一个反思性精炼循环,其中违反的坐标将控制器引导到匹配的修复目标,因此每次迭代仅更改被判断为错误的部分。在有界裁判噪声下,期望的内在差距几何级数收缩到噪声相关的平台。在miniF2F、ProofNet、e-SNLI和ProntoQA上的七个形式化骨干中,精炼持续提升通过率超过单次ICL基线,并且在基线有改进空间的基准上,多轴代理优于匹配的标量代理。因此,结构化代理判断既提供了实用的精炼信号,也提供了在精确参考不可用时收敛的理论依据。
英文摘要:
Autoformalization (AF) requires both prover acceptance and fidelity to an informal source, but prover acceptance alone cannot establish fidelity, and expert-validated reference formalizations are costly to obtain. We introduce a reference-free proxy-judge framework, which separates correctness into grounding, statement faithfulness, coverage, and inference, and a theory of how imperfect checks shape iterative refinement. We establish three results. First, for a fixed sequence of candidates, an added proxy's false-positive and false-negative rates bound its effect on the pass-but-wrong rate. Second, damage to a correct formalization requires a false rejection that exposes it to further refinement, and harmful revisions are then retained through unguarded changes, missed changes, or prover progress that overrides judge warnings. Third, a decomposition of the final pass rate distinguishes continued refinement, generation of new candidates, and their success in passing the checks. We instantiate the framework with eight proxies and test these predictions on ProofNet and miniF2F in Lean~4 and on e-SNLI and ProntoQA in Isabelle/HOL. Under single-proxy removal, no prediction written before the runs is contradicted, and all traced breaks follow the identified failure routes. The results connect proxy reliability to the benefits, risks, and limits of refinement without reference formalizations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。