通过解耦证明者-验证者游戏减轻可读性代价
Mitigating Legibility Tax with Decoupled Prover-Verifier Games
AI总结：
提出解耦证明者-验证者游戏（DPVG），通过分离正确性与可检查性训练一个翻译器模型，将固定求解器的解转化为可检查形式，在保持答案正确性的同时提高可检查性，解决了可读性代价问题。
摘要：
随着大型语言模型能力日益增强，其输出能被能力较弱的系统轻松检查变得至关重要。证明者-验证者游戏可用于提高模型输出的可检查性，但与仅训练以最大化正确性的基线相比，其准确性有所下降——这种现象被称为可读性代价。我们提出一种解决方案，通过将正确性与可检查性条件解耦，转而训练一个“翻译器”模型，将固定求解器模型的解转化为可检查形式。这使我们能够首先训练求解器以最大化正确性，然后训练翻译器将求解器的解翻译成可检查形式，同时保留求解器的答案。为了适应这一新的翻译目标，我们制定了一个解耦的证明者-验证者游戏（DPVG），其均衡对应于忠实且可检查的翻译器。
英文摘要：
As large language models become increasingly capable, it is critical that their outputs can be easily checked by less capable systems. Prover-verifier games can be used to improve checkability of model outputs, but display a degradation in accuracy compared to a baseline trained only to maximize correctness -- a phenonemon named legibility tax. We propose a solution by decoupling the correctness from the checkability condition and instead training a "translator" model that turns a fixed solver model's solution into a checkable form. This allows us to first train the solver to maximize correctness, and then train the translator to translate the solver into a checkable form while retaining the solver's answer. To accommodate this new objective of translation, we formulate a decoupled prover-verifier game (DPVG) where the equilibria correspond to faithful and checkable translators.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
