arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.18702cs.CLcs.AI
收藏

CritiqueLLM:面向大语言模型生成评估的信息丰富型批评生成模型

CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation

Tsinghua University(清华大学);Zhipu AI(智谱AI)

Pei Ke, Bosi Wen, Zhuoer Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang

AI总结:

本文提出Eval-Instruct方法,通过伪参考和多路径提示生成信息丰富型批评数据,微调得到CritiqueLLM模型,在评估性能上超越ChatGPT和开源基线,接近GPT-4,并能作为反馈提升生成质量。

摘要:

自从自然语言处理(NLP)社区开始让大语言模型(LLMs)充当批评家来评估生成文本的质量以来,大多数现有工作都是在由GPT-4直接提示标注的评估数据上训练批评生成模型。我们观察到,这些模型在逐点评分和成对比较中,尤其是在没有参考的情况下,缺乏生成信息丰富型批评的能力。因此,它们生成的批评无法对生成文本提供细粒度的区分度,导致评估性能不尽如人意。在本文中,我们提出了一种简单而有效的方法,称为Eval-Instruct,该方法首先通过伪参考获取逐点评分批评,然后通过多路径提示修改这些批评,以获得不同任务和设置下的信息丰富型评估数据,包括有/无参考的逐点评分和成对比较。在这些数据上微调后,得到的模型CritiqueLLM在实证中显示出优于ChatGPT和所有开源基线,甚至在逐点评分的系统级相关性方面达到了与GPT-4相当的评估性能。我们还证明了我们生成的批评可以作为可扩展的反馈,进一步提高像ChatGPT这样强大LLMs的生成质量。

英文摘要:

Since the natural language processing (NLP) community started to make large language models (LLMs) act as a critic to evaluate the quality of generated texts, most of the existing works train a critique generation model on the evaluation data labeled by GPT-4's direct prompting. We observe that these models lack the ability to generate informative critiques in both pointwise grading and pairwise comparison especially without references. As a result, their generated critiques cannot provide fine-grained distinguishability on generated texts, causing unsatisfactory evaluation performance. In this paper, we propose a simple yet effective method called Eval-Instruct, which can first acquire pointwise grading critiques with pseudo references and then revise these critiques via multi-path prompting to obtain informative evaluation data in different tasks and settings, including pointwise grading and pairwise comparison with / without references. After fine-tuning on these data, the resulting model CritiqueLLM is empirically shown to outperform ChatGPT and all the open-source baselines and even achieve comparable evaluation performance to GPT-4 in system-level correlations of pointwise grading. We also demonstrate that our generated critiques can act as scalable feedback to further improve the generation quality of strong LLMs like ChatGPT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑