困惑度陷阱：基于 PLM 的检索器高估低困惑度文档
Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documents
AI总结：
本文揭示基于 PLM 的检索器因学习困惑度特征而产生来源偏差，并提出推理时去偏方法 CDC，通过分离困惑度偏差效应，在三个领域上验证了其有效性。
摘要：
此前研究发现，基于预训练语言模型（PLM）的检索模型表现出对 LLM 生成内容的偏好，即使这些文档的语义质量与人类撰写的文档相当，也会被赋予更高的相关性分数。这一被称为来源偏差的现象威胁着信息获取生态系统的可持续发展。然而，来源偏差的根本原因尚未被探索。本文利用因果图解释信息检索过程，发现基于 PLM 的检索器学习到了用于相关性估计的困惑度特征，从而将低困惑度文档排在更高位置，导致来源偏差。理论分析进一步揭示，该现象源于语言建模任务与检索任务损失函数梯度之间的正相关关系。基于上述分析，本文提出了一种受因果启发的推理时去偏方法，称为因果诊断与校正（CDC）。CDC 首先诊断困惑度带来的偏差效应，然后将该偏差效应从整体估计的相关性分数中分离出来。在三个领域上的实验结果表明，CDC 具有优越的去偏效果，验证了所提解释框架的有效性。源代码可在 https://github.com/WhyDwelledOnAi/Perplexity-Trap 获取。
英文摘要：
Previous studies have found that PLM-based retrieval models exhibit a preference for LLM-generated content, assigning higher relevance scores to these documents even when their semantic quality is comparable to human-written ones. This phenomenon, known as source bias, threatens the sustainable development of the information access ecosystem. However, the underlying causes of source bias remain unexplored. In this paper, we explain the process of information retrieval with a causal graph and discover that PLM-based retrievers learn perplexity features for relevance estimation, causing source bias by ranking the documents with low perplexity higher. Theoretical analysis further reveals that the phenomenon stems from the positive correlation between the gradients of the loss functions in language modeling task and retrieval task. Based on the analysis, a causal-inspired inference-time debiasing method is proposed, called Causal Diagnosis and Correction (CDC). CDC first diagnoses the bias effect of the perplexity and then separates the bias effect from the overall estimated relevance score. Experimental results across three domains demonstrate the superior debiasing effectiveness of CDC, emphasizing the validity of our proposed explanatory framework. Source codes are available at https://github.com/WhyDwelledOnAi/Perplexity-Trap.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
