Domain-specific finetuning is essential for dense retrievers, yet not all data pairs contribute equally to the learning process. We introduce OPERA, a data pruning framework that exploits this heterogeneity to improve both the effectiveness and efficiency of retrieval model adaptation. We first investigate static pruning (SP), which retains only high-similarity query-document pairs, revealing an intrinsic quality-coverage tradeoff: ranking (NDCG) improves while retrieval (Recall) can degrade due to reduced query diversity. To resolve this tradeoff, we propose a two-stage dynamic pruning (DP) strategy that adaptively modulates sampling probabilities at both query and document levels throughout training, prioritizing high-quality examples while maintaining access to the full training set. Evaluations across eight datasets spanning six domains demonstrate the effectiveness of both approaches: SP improves ranking over standard finetuning (NDCG@10 +0.2 points), while DP achieves the strongest performance on both ranking (NDCG@10 +1.0 points) and retrieval (Recall@20 +0.4 points), with an average rank of 1.38 across all methods. These findings scale to Qwen3-Embedding, an LLM-based dense retriever, confirming architecture-agnostic benefits. Notably, DP reaches comparable performance in less than 50\% of the training time required by standard finetuning.
Safe Pruning LoRA: Robust Distance-Guided Pruning for Safety Alignment in Adaptation of LLMs
School of Electronics and Computer Science, University of Southampton, UK(索姆塞特大学电子与计算机科学学院);Department of Computer Science, University of Liverpool, UK(利弗pool大学计算机科学系)
A Free Lunch in LLM Compression: Revisiting Retraining after Pruning
Department for AI in Society, Science, and Technology, Zuse Institute Berlin(人工智能社会、科学与技术系,柏林Zuse研究所);Institute of Mathematics, Technische Universität Berlin(数学系,柏林技术大学)
All-in-One Tuning and Structural Pruning for Domain-Specific LLMs
Northeastern University(东北大学);George Mason University(乔治梅森大学);GE HealthCare(GE医疗);University of Pennsylvania(宾夕法尼亚大学);University of Pittsburgh(匹兹堡大学);University of Florida(佛罗里达大学);Florida State University(佛罗里达州立大学)
PAT: Pruning-Aware Tuning for Large Language Models
School of Electronic Science and Engineering, Nanjing University(南京大学电子科学与工程学院);University of Arizona(亚利桑那大学);Samsung Electronic Research Centre of China(三星中国电子研究中心);Interdisciplinary Research Center for Future Intelligent Chips, Nanjing University(南京大学未来智能芯片交叉研究中心)
Efficient Post-Training Pruning of Large Language Models with Statistical Correction
Department of Computer Science and Engineering, Santa Clara University, Santa Clara, CA, USA(计算机科学与工程系,圣克拉拉大学);Futurewei Technologies, Inc., USA(未来韦艾技术公司)
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Hong Kong Baptist Univeristy(香港浸会大学);Harbin Institude of Technology, Shenzhen(哈尔滨工业大学(深圳))