arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.10079cs.CL
收藏

迷失在中间与中间地带:增强语言模型在多跳问答中对长上下文的推理能力

Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA

University of Colorado Boulder(科罗拉多大学博尔德分校);University of Chicago(芝加哥大学);Johannes Gutenberg University Mainz(美因茨约翰内斯·古滕贝格大学)

George Arthur Baker, Ankush Raut, Sagi Shaier, Lawrence E Hunter, Katharina von der Wense

AI总结:

本研究探究多跳问答场景下长上下文语言模型的“迷失在中间”问题,发现性能随信息与上下文边缘及信息间距离下降,并提出结合知识图谱三元组提取、摘要和思维链提示的缓解方法。

摘要:

以往研究发现,当前的长上下文语言模型无法均衡利用输入中间位置的信息,更偏好位于输入两端的信息片段,这在我们期望模型能同等能力使用输入不同部分的场景中造成了不合理偏差。截至目前,该问题主要仅在关键信息为单一片段的设置下被研究,这促使我们探究当多条必要信息分散在输入中时会发生什么。本研究展示了“迷失在中间”问题在多跳问答场景下的影响——该场景需要对不连贯的文档进行多次推理“跳步”——并表明性能不仅会因信息与上下文边缘的距离而下降,还会因信息片段之间的距离而下降。此外,我们通过知识图谱三元组提取和摘要技术减少冗余文档内容,并使用思维链提示(chain-of-thought prompting)引导模型更彻底地推理,以此实验了缓解该问题的方法。

英文摘要:

Previous work finds that recent long-context language models fail to make equal use of information in the middle of their inputs, preferring pieces of information located at the tail ends which creates an undue bias in situations where we would like models to be equally capable of using different parts of the input. Thus far, the problem has mainly only been considered in settings with single pieces of critical information, leading us to question what happens when multiple necessary pieces of information are spread out over the inputs. Here, we demonstrate the effects of the "lost in the middle" problem in the multi-hop question answering setting -- in which multiple reasoning "hops" over disconnected documents are required -- and show that performance degrades not only with respect to the distance of information from the edges of the context, but also between pieces of information. Additionally, we experiment with means of alleviating the problem by reducing superfluous document contents through knowledge graph triple extraction and summarization, and prompting models to reason more thoroughly using chain-of-thought prompting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑