MAD-Fact：面向大语言模型长文本事实性评估的多智能体辩论框架
MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMs
AI总结：
针对大语言模型长文本输出事实性评估难题，提出整合数据集、多智能体辩论系统与加权指标的MAD-Fact框架，构建中文数据集LongHalluQA，实验验证了框架有效性及不同模型的事实性表现差异。
摘要：
大语言模型（LLMs）的广泛应用引发了人们对其输出事实准确性的严重担忧，在生物医学、法律、教育等高风险领域尤为突出。现有针对短文本的评估方法因长文本存在复杂推理链、交织观点与累积信息，往往难以适用。为解决该问题，我们提出一套整合大规模长文本数据集、多智能体验证机制与加权评估指标的系统性方法。我们构建了中文长文本事实性数据集LongHalluQA，并开发了基于辩论的多智能体验证系统MAD-Fact，还引入事实重要性层级以体现长文本中不同主张的重要性差异。在两个基准上的实验表明，规模更大的LLM通常具备更高的事实一致性，而国产模型在中文内容上表现更优。本研究为评估与提升LLM长文本输出的事实可靠性提供了结构化框架，可为其在敏感领域的安全部署提供指导。
英文摘要：
The widespread adoption of Large Language Models (LLMs) raises critical concerns about the factual accuracy of their outputs, especially in high-risk domains such as biomedicine, law, and education. Existing evaluation methods for short texts often fail on long-form content due to complex reasoning chains, intertwined perspectives, and cumulative information. To address this, we propose a systematic approach integrating large-scale long-form datasets, multi-agent verification mechanisms, and weighted evaluation metrics. We construct LongHalluQA, a Chinese long-form factuality dataset; and develop MAD-Fact, a debate-based multi-agent verification system. We introduce a fact importance hierarchy to capture the varying significance of claims in long-form texts. Experiments on two benchmarks show that larger LLMs generally maintain higher factual consistency, while domestic models excel on Chinese content. Our work provides a structured framework for evaluating and enhancing factual reliability in long-form LLM outputs, guiding their safe deployment in sensitive domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
