通过中间表示优化多跳文档检索
Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
AI总结：
本文提出L-RAG方法，通过利用中间层表示优化多跳文档检索，提升问答性能。
摘要：
检索增强生成（RAG）在处理复杂查询，尤其是多跳问题时面临挑战。虽然几种方法通过迭代生成内部查询并检索外部文档来处理多跳查询，但这些方法计算成本较高。在本文中，我们发现LLMs在逐层推理过程中存在三种阶段的信息处理模式，包括提取、处理和后续提取步骤。这一观察表明，中间层的表示比其他层的表示包含更丰富的信息。基于这一见解，我们提出了逐层RAG（L-RAG）。与以往仅生成新内部查询的方法不同，L-RAG利用中间层的表示，这些表示捕捉到了下一步信息，以检索外部知识。L-RAG在性能上与多步方法相当，同时保持了与标准RAG相似的推理开销。实验结果表明，L-RAG在开放领域多跳问答数据集上优于现有RAG方法，包括MuSiQue、HotpotQA和2WikiMultiHopQA。代码可在https://github.com/Olive-2019/L-RAG获取。
英文摘要：
Retrieval-augmented generation (RAG) encounters challenges when addressing complex queries, particularly multi-hop questions. While several methods tackle multi-hop queries by iteratively generating internal queries and retrieving external documents, these approaches are computationally expensive. In this paper, we identify a three-stage information processing pattern in LLMs during layer-by-layer reasoning, consisting of extraction, processing, and subsequent extraction steps. This observation suggests that the representations in intermediate layers contain richer information compared to those in other layers. Building on this insight, we propose Layer-wise RAG (L-RAG). Unlike prior methods that focus on generating new internal queries, L-RAG leverages intermediate representations from the middle layers, which capture next-hop information, to retrieve external knowledge. L-RAG achieves performance comparable to multi-step approaches while maintaining inference overhead similar to that of standard RAG. Experimental results show that L-RAG outperforms existing RAG methods on open-domain multi-hop question-answering datasets, including MuSiQue, HotpotQA, and 2WikiMultiHopQA. The code is available in https://github.com/Olive-2019/L-RAG
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
