上下文感知解码减少查询导向摘要中的幻觉
Context-aware Decoding Reduces Hallucination in Query-focused Summarization
AI总结：
本文提出上下文感知解码方法，通过减少查询导向摘要中的幻觉并保留词法模式匹配度，提升生成质量。
摘要：
查询导向摘要（QFS）旨在提供一个单一文档或多文档的摘要，以满足给定查询的信息需求。它在各种现实应用中都很有用，例如抽象片段生成或更近期的检索增强生成（RAG）。典型的QFS流程包括检索器（稀疏或密集检索）和生成器（通常是一个大语言模型）。然而，应用大语言模型（LLM）可能会导致幻觉，尤其是在证据与LLM的先前信念相矛盾时。近年来，开发新的解码方法以提高生成质量和减少幻觉引起了广泛关注。在本工作中，我们对一种 recently 提出的解码方法——上下文感知解码（CAD）进行了大规模可重复性研究。除了在新闻摘要数据集上复现CAD的实验外，我们还进行了QFS数据集的实验，并对计算复杂性和超参数敏感性进行了更严格的分析。使用八种不同的语言模型进行的实验表明，从性能角度来看，CAD通过（1）减少事实性错误/幻觉的同时（2）大部分保留了词法模式的匹配度，通过ROUGE分数衡量，同时以增加推理时间FLOPs和减少解码速度为代价。基于Huggingface库的代码实现已公开。
英文摘要：
Query-focused summarization (QFS) aims to provide a summary of a single document/multi documents that can satisfy the information needs of a given query. It is useful for various real-world applications, such as abstractive snippet generation or more recent retrieval augmented generation (RAG). A prototypical QFS pipeline consists of a retriever (sparse or dense retrieval) and a generator (usually a large language model). However, applying large language models (LLM) potentially leads to hallucinations, especially when the evidence contradicts the prior belief of LLMs. There has been growing interest in developing new decoding methods to improve generation quality and reduce hallucination. In this work, we conduct a large-scale reproducibility study on one recently proposed decoding method\, -- \,Context-aware Decoding (CAD). In addition to replicating CAD's experiments on news summarization datasets, we include experiments on QFS datasets, and conduct more rigorous analysis on computational complexity and hyperparameter sensitivity. Experiments with eight different language models show that performance-wise, CAD improves QFS quality by (1) reducing factuality errors/hallucinations while (2) mostly retaining the match of lexical patterns, measured by ROUGE scores, while also at a cost of increased inference-time FLOPs and reduced decoding speed. The \href{https://github.com/zhichaoxu-shufe/context-aware-decoding-qfs}{code implementation} based on Huggingface Library is made available
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
