RetroLLM：赋能大型语言模型在生成过程中检索细粒度证据
RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation
AI总结：
RetroLLM提出统一框架，将检索与生成整合，通过层次化FM-Index约束和前瞻性约束解码，使LLM直接生成细粒度证据，在五个开放域问答数据集上表现优越。
摘要：
大型语言模型（LLMs）展现出卓越的生成能力，但常常出现幻觉问题。检索增强生成（RAG）通过引入外部知识提供了一种有效的解决方案，但现有方法仍面临若干局限：额外部署独立检索器的成本、检索文本块带来的冗余输入标记，以及缺乏检索与生成的联合优化。为解决这些问题，我们提出了RetroLLM，一个将检索与生成整合为单一连贯过程的统一框架，使LLMs能够通过约束解码直接从语料库中生成细粒度证据。此外，为缓解约束证据生成过程中的错误剪枝问题，我们引入了（1）层次化FM-Index约束，该约束生成语料库约束线索以在证据生成前识别相关文档子集，从而减少无关解码空间；以及（2）前瞻性约束解码策略，该策略考虑未来序列的相关性以提高证据准确性。在五个开放域问答数据集上的大量实验表明，RetroLLM在域内和域外任务上均表现出优越性能。代码可在https://github.com/sunnynexus/RetroLLM获取。
英文摘要：
Large language models (LLMs) exhibit remarkable generative capabilities but often suffer from hallucinations. Retrieval-augmented generation (RAG) offers an effective solution by incorporating external knowledge, but existing methods still face several limitations: additional deployment costs of separate retrievers, redundant input tokens from retrieved text chunks, and the lack of joint optimization of retrieval and generation. To address these issues, we propose \textbf{RetroLLM}, a unified framework that integrates retrieval and generation into a single, cohesive process, enabling LLMs to directly generate fine-grained evidence from the corpus with constrained decoding. Moreover, to mitigate false pruning in the process of constrained evidence generation, we introduce (1) hierarchical FM-Index constraints, which generate corpus-constrained clues to identify a subset of relevant documents before evidence generation, reducing irrelevant decoding space; and (2) a forward-looking constrained decoding strategy, which considers the relevance of future sequences to improve evidence accuracy. Extensive experiments on five open-domain QA datasets demonstrate RetroLLM's superior performance across both in-domain and out-of-domain tasks. The code is available at \url{https://github.com/sunnynexus/RetroLLM}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
