WebCiteS：带引用的中文网页搜索结果查询聚焦摘要
WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citations
AI总结：
本文提出中文数据集WebCiteS（含7000条带引用标注），定义查询聚焦摘要任务，并开发细粒度指标自动区分基础性错误与引用错误，评估显示LLM引用来源仍具挑战。
摘要：
增强大型语言模型（LLMs）中的归因能力是一项关键任务。一种可行的方法是使LLMs能够引用支持其生成内容的外部来源。然而，该领域现有的数据集和评估方法仍存在显著局限性。在这项工作中，我们定义了带引用的查询聚焦摘要（AQFS）任务，并提出了WebCiteS，一个包含7000条带引用的人工标注摘要的中文数据集。WebCiteS源自真实用户查询和网页搜索结果，为模型训练和评估提供了宝贵资源。以往在归因评估方面的研究未能区分基础性错误和引用错误，也未能自动验证部分支持来自多个来源的句子。我们通过开发详细指标并让自动评估器将句子分解为子声明进行细粒度验证来解决这些问题。我们在WebCiteS上对开源和专有模型进行的全面评估凸显了LLMs在正确引用来源方面面临的挑战，强调了进一步改进的必要性。数据集和代码将开源，以促进这一关键领域的进一步研究。
英文摘要：
Enhancing the attribution in large language models (LLMs) is a crucial task. One feasible approach is to enable LLMs to cite external sources that support their generations. However, existing datasets and evaluation methods in this domain still exhibit notable limitations. In this work, we formulate the task of attributed query-focused summarization (AQFS) and present WebCiteS, a Chinese dataset featuring 7k human-annotated summaries with citations. WebCiteS derives from real-world user queries and web search results, offering a valuable resource for model training and evaluation. Prior works in attribution evaluation do not differentiate between groundedness errors and citation errors. They also fall short in automatically verifying sentences that draw partial support from multiple sources. We tackle these issues by developing detailed metrics and enabling the automatic evaluator to decompose the sentences into sub-claims for fine-grained verification. Our comprehensive evaluation of both open-source and proprietary models on WebCiteS highlights the challenge LLMs face in correctly citing sources, underscoring the necessity for further improvement. The dataset and code will be open-sourced to facilitate further research in this crucial field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
