arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.17205cs.IRcs.AIcs.CLcs.LG
收藏

OPERA:面向高效检索模型适应的在线数据剪枝

OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation

Amazon Web Services(亚马逊云服务)

Haoyang Fang, Shuai Zhang, Yifei Ma, Hengyi Wang, Cuixiong Hu, Katrin Kirchhoff, Bernie Wang, George Karypis

AI总结:

OPERA通过在线数据剪枝提升检索模型适应效果,SP在标准微调基础上提升NDCG,DP在多个领域数据集上取得最佳排名和召回效果,且训练时间减少50%。

摘要:

领域特定微调对密集检索器至关重要,但并非所有训练对对学习过程贡献相同。我们介绍了OPERA,一种数据剪枝框架,利用这种异质性提升检索模型适应的有效性和效率。我们首先研究了静态剪枝(SP),仅保留高相似度的查询-文档对,揭示了内在的质量-覆盖权衡:排名(NDCG)提高而检索(Recall)可能因查询多样性减少而下降。为解决这一权衡,我们提出了一种两阶段动态剪枝(DP)策略,通过训练过程中在查询和文档层面动态调节采样概率,优先选择高质量示例同时保持完整训练集的访问。在八个跨六个领域的数据集上评估显示,两种方法均有效:SP在标准微调基础上提升NDCG(NDCG@10 +0.5%),而DP在排名(NDCG@10 +1.9%)和检索(Recall@20 +0.7%)上表现最佳,所有方法的平均排名为1.38。这些发现扩展到Qwen3-Embedding,一种基于大语言模型的密集检索器,确认了架构无关的益处。值得注意的是,DP在标准微调所需训练时间的50%内达到可比性能。

英文摘要:

Domain-specific finetuning is essential for dense retrievers, yet not all data pairs contribute equally to the learning process. We introduce OPERA, a data pruning framework that exploits this heterogeneity to improve both the effectiveness and efficiency of retrieval model adaptation. We first investigate static pruning (SP), which retains only high-similarity query-document pairs, revealing an intrinsic quality-coverage tradeoff: ranking (NDCG) improves while retrieval (Recall) can degrade due to reduced query diversity. To resolve this tradeoff, we propose a two-stage dynamic pruning (DP) strategy that adaptively modulates sampling probabilities at both query and document levels throughout training, prioritizing high-quality examples while maintaining access to the full training set. Evaluations across eight datasets spanning six domains demonstrate the effectiveness of both approaches: SP improves ranking over standard finetuning (NDCG@10 +0.2 points), while DP achieves the strongest performance on both ranking (NDCG@10 +1.0 points) and retrieval (Recall@20 +0.4 points), with an average rank of 1.38 across all methods. These findings scale to Qwen3-Embedding, an LLM-based dense retriever, confirming architecture-agnostic benefits. Notably, DP reaches comparable performance in less than 50\% of the training time required by standard finetuning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑