Debatrix：基于大语言模型的具备迭代时序分析能力的多维度辩论裁判
Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM
AI总结：
针对多轮长辩论自动化评估的难题，提出融合迭代时序分析与多维度评估协同的Debatrix框架，结合PanelBench基准验证，其效果显著优于直接用大语言模型评估。
摘要：
我们如何构建一个自动化辩论裁判，来评估篇幅长、活跃度高的多轮辩论？这一任务颇具挑战性，因为辩论评判需要处理长文本、复杂的论点关联以及多维度评估。同时，当前研究主要聚焦于短对话，极少涉及对整场辩论的评估。本文中，我们借助大语言模型（Large Language Models, LLM），提出Debatrix，使多轮辩论的分析与评估更贴合多数人的偏好。具体而言，Debatrix具备纵向的迭代时序分析与横向的多维度评估协同两大特性。为贴合真实辩论场景，我们引入了PanelBench基准，将系统表现与实际辩论结果进行对比。结果表明，该方法相比直接使用LLM进行辩论评估有显著提升。源代码与基准数据可在https://github.com/ljcleo/debatrix 获取。
英文摘要：
How can we construct an automated debate judge to evaluate an extensive, vibrant, multi-turn debate? This task is challenging, as judging a debate involves grappling with lengthy texts, intricate argument relationships, and multi-dimensional assessments. At the same time, current research mainly focuses on short dialogues, rarely touching upon the evaluation of an entire debate. In this paper, by leveraging Large Language Models (LLMs), we propose Debatrix, which makes the analysis and assessment of multi-turn debates more aligned with majority preferences. Specifically, Debatrix features a vertical, iterative chronological analysis and a horizontal, multi-dimensional evaluation collaboration. To align with real-world debate scenarios, we introduced the PanelBench benchmark, comparing our system's performance to actual debate outcomes. The findings indicate a notable enhancement over directly using LLMs for debate evaluation. Source code and benchmark data are available online at https://github.com/ljcleo/debatrix .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
