SAGEval：面向无参考开放式文本的基于满意智能体的自然语言生成评估前沿
SAGEval: The frontiers of Satisfactory Agent based NLG Evaluation for reference-free open-ended text
AI总结：
本文提出SAGEval框架，利用评判智能体纠正LLM评估器分数，在无参考标签条件下实现开放式文本生成评估，减少对标注数据的依赖。
摘要：
大型语言模型（LLM）集成到Microsoft365套件和Google Workspace等应用中，用于创建/处理文档、电子邮件、演示文稿等，已显著提升生产力并节省时间。但随着这些集成变得越来越复杂，确保LLM集成应用输出内容的质量具有相关性和适用性至关重要。鉴于需要为自然语言生成开发稳健的评估方法，而在这些场景中参考/真实标签不存在或不够充分，本文提出了一种名为“SAGEval”的新框架，该框架利用一个评判智能体对LLM评估器生成的分数提供反馈。我们表明，在缺少参考/真实标签的情况下，评判智能体能够纠正LLM评估器的分数，从而减少对标注数据的需求，即使对于复杂的NLG评估场景也是如此，例如生成JSON结构化的表单/问卷，其回答包含多种风格，如多项选择、李克特评分、单选题等。
英文摘要：
Large Language Model (LLM) integrations into applications like Microsoft365 suite and Google Workspace for creating/processing documents, emails, presentations, etc. has led to considerable enhancements in productivity and time savings. But as these integrations become more more complex, it is paramount to ensure that the quality of output from the LLM-integrated applications are relevant and appropriate for use. Identifying the need to develop robust evaluation approaches for natural language generation, wherein references/ground labels doesn't exist or isn't amply available, this paper introduces a novel framework called "SAGEval" which utilizes a critiquing Agent to provide feedback on scores generated by LLM evaluators. We show that the critiquing Agent is able to rectify scores from LLM evaluators, in absence of references/ground-truth labels, thereby reducing the need for labeled data even for complex NLG evaluation scenarios, like the generation of JSON-structured forms/surveys with responses in different styles like multiple choice, likert ratings, single choice questions, etc.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
