从单文档到多文档：大语言模型在多文档摘要中如何产生幻觉
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization
AI总结：
本研究探讨了LLMs在多文档摘要（MDS）任务中的幻觉表现，创建了两个新基准，发现生成摘要平均75%内容为幻觉，且事后基线方法仅能中等程度缓解幻觉。
摘要：
尽管许多研究已经调查并减少了单文档任务中大语言模型（LLMs）的幻觉，但关于多文档摘要（MDS）任务中幻觉的研究在很大程度上仍未被探索。具体而言，目前尚不清楚处理多个文档（例如，信息的重复性和多样性）所带来的挑战如何影响模型输出。在本研究中，我们调查了当 LLMs 从多个文档中总结特定主题信息时，幻觉是如何表现的。由于目前没有用于调查 MDS 中幻觉的基准，我们使用带有特定主题见解标注的现有新闻和对话数据集，创建了两个新颖的多文档基准。当在我们的基准上评估 5 个 LLMs 时，我们观察到，LLM 生成的摘要中平均高达 75% 的内容是幻觉，且幻觉更可能出现在摘要的末尾。此外，当总结不存在的主题相关信息时，gpt-3.5-turbo 和 GPT-4o 仍然分别在约 79.35% 和 44% 的情况下生成摘要，这引发了对其捏造内容倾向的担忧。为了理解这些幻觉的特征，我们手动评估了 700 多条见解，发现大多数错误源于未能遵循指令或产生过于笼统的见解。受这些观察的启发，我们调查了简单的事后基线方法在减轻幻觉方面的有效性，但发现它们仅具有中等效果。我们的结果强调，需要更有效的方法来系统地减轻 MDS 中的幻觉。我们在 github.com/megagonlabs/Hallucination_MDS 发布了我们的数据集和代码。
英文摘要：
Although many studies have investigated and reduced hallucinations in large language models (LLMs) for single-document tasks, research on hallucination in multi-document summarization (MDS) tasks remains largely unexplored. Specifically, it is unclear how the challenges arising from handling multiple documents (e.g., repetition and diversity of information) affect models outputs. In this work, we investigate how hallucinations manifest in LLMs when summarizing topic-specific information from multiple documents. Since no benchmarks exist for investigating hallucinations in MDS, we use existing news and conversation datasets, annotated with topic-specific insights, to create two novel multi-document benchmarks. When evaluating 5 LLMs on our benchmarks, we observe that on average, up to 75% of the content in LLM-generated summary is hallucinated, with hallucinations more likely to occur towards the end of the summaries. Moreover, when summarizing non-existent topic-related information, gpt-3.5-turbo and GPT-4o still generate summaries about 79.35% and 44% of the time, raising concerns about their tendency to fabricate content. To understand the characteristics of these hallucinations, we manually evaluate 700+ insights and find that most errors stem from either failing to follow instructions or producing overly generic insights. Motivated by these observations, we investigate the efficacy of simple post-hoc baselines in mitigating hallucinations but find them only moderately effective. Our results underscore the need for more effective approaches to systematically mitigate hallucinations in MDS. We release our dataset and code at github.com/megagonlabs/Hallucination_MDS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
