FRESCO：用于检索增强生成中动态语义冲突的重排序器基准测试与优化
FRESCO: Benchmarking and Optimizing Re-rankers for Evolving Semantic Conflict in Retrieval-Augmented Generation
AI总结：
FRESCO针对检索增强生成中动态语义冲突问题，评估重排序器在时间动态场景下的性能，发现现有重排序器对较旧文档存在偏见，通过指令优化框架改进了27%的动态知识任务性能。
摘要：
检索增强生成（RAG）通过将响应基于最新证据来缓解大型语言模型的时间陈旧问题。在RAG流程中，重排序器在选择最有用的文档中起关键作用。然而，现有基准主要评估重排序器在静态环境中的表现，未能充分评估在信息演变下的性能——这在现实系统中至关重要，因为系统必须在时间不同的证据中进行选择。为了解决这一限制，我们引入FRESCO（事实近期性和演变语义冲突），用于评估重排序器在时间动态环境中的性能。通过将寻求近期性的查询与历史维基百科修订配对，FRESCO测试重排序器是否能在优先选择事实性近期证据的同时保持语义相关性。我们的评估揭示了现有重排序器的一致性失败模式：即使文档事实性过时，仍存在对较旧、语义丰富的文档的偏见。我们进一步研究了一个指令优化框架以缓解这一问题。通过识别平衡演变和非演变知识任务的帕累托最优指令，我们获得了在演变知识任务上高达27%的改进，同时在非演变知识任务上保持竞争性性能。
英文摘要：
Retrieval-Augmented Generation (RAG) is a key approach to mitigating the temporal staleness of large language models (LLMs) by grounding responses in up-to-date evidence. Within the RAG pipeline, re-rankers play a pivotal role in selecting the most useful documents from retrieved candidates. However, existing benchmarks predominantly evaluate re-rankers in static settings and do not adequately assess performance under evolving information -- a critical gap, as real-world systems often must choose among temporally different pieces of evidence. To address this limitation, we introduce FRESCO (Factual Recency and Evolving Semantic COnflict), a benchmark for evaluating re-rankers in temporally dynamic contexts. By pairing recency-seeking queries with historical Wikipedia revisions, FRESCO tests whether re-rankers can prioritize factually recent evidence while maintaining semantic relevance. Our evaluation reveals a consistent failure mode across existing re-rankers: a strong bias toward older, semantically rich documents, even when they are factually obsolete. We further investigate an instruction optimization framework to mitigate this issue. By identifying Pareto-optimal instructions that balance Evolving and Non-Evolving Knowledge tasks, we obtain gains of up to 27% on Evolving Knowledge tasks while maintaining competitive performance on Non-Evolving Knowledge tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
