arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.01830cs.CLcs.AIcs.LG
收藏

PiCO:基于一致性优化的大语言模型同行评审

PiCO: Peer Review in LLMs based on the Consistency Optimization

School of Electrical and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Peng Cheng Laboratory(鹏城实验室)

Kun-Peng Ning, Shuo Yang, Yu-Yang Liu, Jia-Yu Yao, Zhen-Hui Liu, Yong-Hong Tian, Yibing Song, Li Yuan

AI总结:

该研究提出基于一致性优化的LLM无监督同行评审方法PiCO,通过可学习能力参数构建能力层级优化模型,并设计三项指标验证其在多数据集上的评估有效性。

摘要:

现有大语言模型(LLM)评估方法通常聚焦于在带人工标注的封闭环境、特定领域基准上测试性能。本文探索了一种全新的无监督评估方向,利用同行评审机制自动衡量LLM能力。在该设置下,开源与闭源LLM处于同一环境,可回答无标注问题并互相评估,每个LLM的回答得分由其他匿名LLM共同决定。为获取这些模型的能力层级,我们为每个LLM分配一个可学习的能力参数以调整最终排名,并将其形式化为一个带约束的优化问题,目标是最大化每个LLM的能力与得分的一致性。其背后的核心假设是:高水平LLM能比低水平LLM更准确地评估他人的回答,同时高水平LLM也能获得更高的回答得分。此外,我们提出了PEN、CIN和LIS三个指标,用于评估与人类排名对齐的差距。我们在多个数据集上采用这些指标开展实验,验证了所提方法的有效性。

英文摘要:

Existing large language models (LLMs) evaluation methods typically focus on testing the performance on some closed-environment and domain-specific benchmarks with human annotations. In this paper, we explore a novel unsupervised evaluation direction, utilizing peer-review mechanisms to measure LLMs automatically. In this setting, both open-source and closed-source LLMs lie in the same environment, capable of answering unlabeled questions and evaluating each other, where each LLM's response score is jointly determined by other anonymous ones. To obtain the ability hierarchy among these models, we assign each LLM a learnable capability parameter to adjust the final ranking. We formalize it as a constrained optimization problem, intending to maximize the consistency of each LLM's capabilities and scores. The key assumption behind is that high-level LLM can evaluate others' answers more accurately than low-level ones, while higher-level LLM can also achieve higher response scores. Moreover, we propose three metrics called PEN, CIN, and LIS to evaluate the gap in aligning human rankings. We perform experiments on multiple datasets with these metrics, validating the effectiveness of the proposed approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑