arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.09770cs.CL
收藏

黄金矿砂:用于针叶堆寻针推理的战略上下文洗牌

GOLD PANNING: Strategic Context Shuffling for Needle-in-Haystack Reasoning

Johns Hopkins University(约翰霍普金斯大学)

Adam Byerly, Daniel Khashabi

AI总结:

GOLD PANNING通过战略上下文洗牌方法,在针叶堆推理中以更少的查询实现高效目标识别,利用信号锚定技术提升模型性能。

摘要:

大型语言模型(LLMs)在长上下文针叶堆问题中表现出显著的位置偏差,系统性地优先考虑信息的位置而非相关性。尽管当前的缓解措施依赖于白盒访问,但对许多最先进的模型来说这实际上是不可能的。我们介绍了GOLD PANNING,一种黑盒贝叶斯框架,通过在长上下文中进行推理时间的主动搜索来执行(i)重新排列文档以将高信念项集中在高度诊断性位置(信号锚定)以及(ii)从模型输出更新文档相关性的信念。与传统的主动学习不同,GOLD PANNING利用锚定——一旦标记,就保持在视线中——来保留弱提示。我们通过迭代分配实现这一点,该分配源自模型的诊断性轮廓,可以证明在N个文档中以O(log N)轮次识别目标,确保可扩展性到多文档设置。在针叶堆检索和长上下文问答中,GOLD PANNING在目标识别方面与Permutation Self-Consistency相当,但查询数量减少了30-65%,并在校准不匹配的情况下仍然有效,表明粗略的位置排序驱动了性能提升。这些结果表明,固有的模型偏差不必是失败,但可以作为控制工具。

英文摘要:

Large language models (LLMs) exhibit pronounced position bias in long-context needle-in-haystack problems, systematically prioritizing the location of information over its relevance. While current mitigations rely on white-box access, this is effectively impossible for many state-of-the-art models. We introduce GOLD PANNING, a black-box Bayesian framework that performs inference-time active search over long contexts by (i) reordering documents to concentrate high-belief items in highly diagnostic positions (signal anchoring) and (ii) updating beliefs over document relevance from model outputs. Unlike conventional active learning, which prioritizes uncertainty reduction, GOLD PANNING leverages anchoring -- once flagged, keep it in sight -- to preserve weak cues. We implement this using iterative assignment derived from the model's diagnosticity profile, which provably identifies a target among $N$ documents in $O(\log N)$ rounds, ensuring scalability to many-document settings.On needle-in-a-haystack retrieval and long-context QA, GOLD PANNING matches Permutation Self-Consistency's target identification with $30--65%$ fewer queries and remains effective under calibration mismatch, suggesting coarse positional ordering drives performance gains. These results demonstrate that inherent model biases need not be failures, but can be used as tools for control.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑