arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.15723cs.AI
收藏

问答模型中的上下文长度鲁棒性:一项比较实证研究

Context-Length Robustness in Question Answering Models: A Comparative Empirical Study

Upper Hand;Ace Rent a Car

Trishita Dhara, Siddhesh Sheth

AI总结:

研究通过SQuAD和HotpotQA基准评估问答模型在上下文长度变化下的鲁棒性,发现多跳推理任务比单跨度提取任务更易受上下文稀释影响。

摘要:

大型语言模型越来越多地应用于相关信息嵌入在长而嘈杂上下文中的场景。尽管如此,不同问答任务中对增长上下文长度的鲁棒性仍理解不足。本文通过SQuAD和HotpotQA两个广泛使用的基准,进行了受控的实证研究,评估模型准确性随总上下文长度的变化。通过系统性地增加无关上下文并保持答案信号,我们能够将上下文长度的影响与任务难度变化分离。结果表明,随着上下文长度增加,性能持续下降,多跳推理任务比单跨度提取任务表现出显著更大的性能下降。特别是,HotpotQA在等效上下文扩展下表现出几乎两倍的准确性下降。这些发现突显了任务依赖的鲁棒性差异,并表明多跳推理特别容易受到上下文稀释的影响。我们主张在评估模型可靠性时应明确评估上下文长度鲁棒性,尤其对于涉及长文档或检索增强生成的应用。

英文摘要:

Large language models are increasingly deployed in settings where relevant information is embedded within long and noisy contexts. Despite this, robustness to growing context length remains poorly understood across different question answering tasks. In this work, we present a controlled empirical study of context-length robustness in large language models using two widely used benchmarks: SQuAD and HotpotQA. We evaluate model accuracy as a function of total context length by systematically increasing the amount of irrelevant context while preserving the answer-bearing signal. This allows us to isolate the effect of context length from changes in task difficulty. Our results show a consistent degradation in performance as context length increases, with substantially larger drops observed on multi-hop reasoning tasks compared to single-span extraction tasks. In particular, HotpotQA exhibits nearly twice the accuracy degradation of SQuAD under equivalent context expansions. These findings highlight task-dependent differences in robustness and suggest that multi-hop reasoning is especially vulnerable to context dilution. We argue that context-length robustness should be evaluated explicitly when assessing model reliability, especially for applications involving long documents or retrieval-augmented generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑