CREAM：面向会议摘要的基于比较的无参考ELO排名自动评估方法
CREAM: Comparison-Based Reference-Free ELO-Ranked Automatic Evaluation for Meeting Summarization
AI总结：
针对现有自动评估方法在会议摘要等复杂任务上效果不佳的问题，本文提出无参考评估框架CREAM，结合思维链与关键事实对齐评估摘要质量，并通过ELO排名实现不同模型或提示配置的鲁棒比较。
摘要：
大语言模型（LLMs）推动了摘要自动评估方法的研究热潮，这类方法相比人工评估速度更快、成本更低。然而，现有方法应用于长上下文摘要、对话式会议摘要等复杂任务时往往效果不佳。本文提出CREAM（基于比较的无参考ELO排名会议摘要自动评估）这一新型框架，以应对会议摘要评估的独特挑战。CREAM结合思维链推理与关键事实对齐技术，无需参考摘要即可评估模型生成摘要的简洁性与完整性。通过采用ELO排名系统，该方法为比较不同模型或提示配置的质量提供了鲁棒的机制。
英文摘要：
Large Language Models (LLMs) have spurred interest in automatic evaluation methods for summarization, offering a faster, more cost-effective alternative to human evaluation. However, existing methods often fall short when applied to complex tasks like long-context summarizations and dialogue-based meeting summarizations. In this paper, we introduce CREAM (Comparison-Based Reference-Free Elo-Ranked Automatic Evaluation for Meeting Summarization), a novel framework that addresses the unique challenges of evaluating meeting summaries. CREAM leverages a combination of chain-of-thought reasoning and key facts alignment to assess conciseness and completeness of model-generated summaries without requiring reference. By employing an ELO ranking system, our approach provides a robust mechanism for comparing the quality of different models or prompt configurations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
