通过重复错序上下文释放大型语言模型中的多跳推理潜力
Unleashing Multi-Hop Reasoning Potential in Large Language Models through Repetition of Misordered Context
AI总结：
本文提出上下文重复（CoRe）方法，通过重复呈现上下文解决LLMs多跳推理中的错序上下文问题，提升F1分数最高30%p，准确率最高70%p，并缓解“中间迷失”问题。
摘要：
多跳推理要求基于给定上下文中的支持文档进行多步推理，这对大型语言模型（LLMs）仍具挑战性。LLMs常难以过滤上下文中的无关文档，且其性能对支持文档在上下文中的绝对位置敏感。本文识别出另一挑战：LLMs的性能也对支持文档呈现的顺序（即相对位置）敏感，我们称之为错序上下文问题。为解决此问题，基于理论方法，我们提出一种简单有效的方法——上下文重复（CoRe），通过重复呈现上下文来提示模型。这确保支持文档中某些连续的推理片段以最优顺序呈现，有效引导模型朝正确方向推理。应用CoRe后，我们在多跳QA任务上将F1分数提升最高30%p，在合成任务上将准确率提升最高70%p。此外，CoRe有助于缓解LLMs中著名的“中间迷失”问题，并能与利用思维链推理的检索方法有效结合。
英文摘要：
Multi-hop reasoning, which requires multi-step reasoning based on the supporting documents within a given context, remains challenging for large language models (LLMs). LLMs often struggle to filter out irrelevant documents within the context, and their performance is sensitive to the absolute position of supporting documents within that context. In this paper, we identify an additional challenge: LLMs' performance is also sensitive to the order, relative position, in which the supporting documents are presented. We refer to this as the misordered context problem. To address this issue, based on the theoretical approach, we propose a simple yet effective method called context repetition (CoRe), which involves prompting the model by repeatedly presenting the context. This ensures that certain contiguous reasoning segments within supporting documents are presented in the optimal order, effectively guiding the model's reasoning in the appropriate direction. Applying CoRe, we improve the F1 score by up to 30%p on multi-hop QA tasks and increase accuracy by up to 70%p on a synthetic task. Additionally, CoRe helps mitigate the well-known "lost-in-the-middle" problem in LLMs and can be effectively combined with retrieval-based approaches utilizing Chain-of-Thought (CoT) reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
