FUSE:无需标注数据的验证器集成
FUSE: Ensembling Verifiers with Zero Labeled Data
AI总结:
FUSE通过无监督集成验证器提升大语言模型输出验证质量,无需标注数据,在多种生成模型和基准测试中表现优异。
摘要:
模型输出验证正成为训练和实际部署大型语言模型(LLMs)的关键基础。实践中,这通常涉及使用不完美的LLM评判者和奖励模型,因为获取真实标签耗时且昂贵。我们介绍了完全无监督评分集成(FUSE),一种通过集成验证器提高验证质量的方法,无需访问真实正确性标签。FUSE的核心思想是通过控制验证器之间的条件依赖性,改进从集成文献中的一类谱算法的无监督性能。尽管不需要真实标签,FUSE在与不同生成模型、验证器和基准测试的测试时间扩展实验中通常匹配或优于半监督替代方法。特别是,我们在传统学术基准如GPQA Diamond以及前沿、未饱和的基准如Humanity's Last Exam和IMO Shortlist问题上验证了我们的方法。
英文摘要:
Verification of model outputs is rapidly emerging as a key primitive for both training and real-world deployment of large language models (LLMs). In practice, this often involves using imperfect LLM judges and reward models since ground truth acquisition can be time-consuming and expensive. We introduce Fully Unsupervised Score Ensembling (FUSE), a method for improving verification quality by ensembling verifiers without access to ground truth correctness labels. The key idea behind FUSE is to control conditional dependencies between verifiers in a manner that improves the unsupervised performance of a class of spectral algorithms from the ensembling literature. Despite requiring zero ground truth labels, FUSE typically matches or improves upon semi-supervised alternatives in test-time scaling experiments with diverse sets of generator models, verifiers, and benchmarks. In particular, we validate our method on both conventional academic benchmarks such as GPQA Diamond and on frontier, unsaturated benchmarks such as Humanity's Last Exam and IMO Shortlist questions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。