稠密检索器的崩溃：短文档、靠前位置与字面匹配偏差压过事实证据
Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence
AI总结：
本文通过受控实验揭示稠密检索器存在短文档、靠前位置、重复实体和字面匹配等启发式偏差，多种偏差叠加时性能灾难性下降，并证明这些偏差会误导 RAG 中的大语言模型，使性能比不提供文档时下降 34%。
摘要：
稠密检索模型广泛用于信息检索（IR）应用中，例如检索增强生成（RAG）。由于它们通常是这些系统的第一步，其鲁棒性对于避免下游失败至关重要。在本工作中，我们重新利用一个关系抽取数据集（如 Re-DocRED）来设计受控实验，以量化启发式偏差（例如对较短文档的偏好）对 Dragon+ 和 Contriever 等检索器的影响。我们揭示了重大脆弱性，表明检索器偏好较短文档、靠前位置、重复实体和字面匹配，同时完全忽略答案的存在！值得注意的是，当多种偏差叠加时，模型表现出灾难性的性能下降：在一个不含答案的合成偏差文档面前，选择含答案文档的比例不到 10%。此外，我们表明这些偏差对 RAG 等下游应用有直接后果，检索器偏好的文档可能误导大语言模型（LLM），导致性能比完全不提供文档时下降 34%。https://huggingface.co/datasets/mohsenfayyaz/ColDeR
英文摘要：
Dense retrieval models are commonly used in Information Retrieval (IR) applications, such as Retrieval-Augmented Generation (RAG). Since they often serve as the first step in these systems, their robustness is critical to avoid downstream failures. In this work, we repurpose a relation extraction dataset (e.g., Re-DocRED) to design controlled experiments that quantify the impact of heuristic biases, such as a preference for shorter documents, on retrievers like Dragon+ and Contriever. We uncover major vulnerabilities, showing retrievers favor shorter documents, early positions, repeated entities, and literal matches, all while ignoring the answer's presence! Notably, when multiple biases combine, models exhibit catastrophic performance degradation, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer. Furthermore, we show that these biases have direct consequences for downstream applications like RAG, where retrieval-preferred documents can mislead LLMs, resulting in a 34% performance drop than providing no documents at all. https://huggingface.co/datasets/mohsenfayyaz/ColDeR
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
