arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.06166cs.LG
收藏

一个算法,两个目标:在LLM微调中用于参数和数据选择的双评分

One Algorithm, Two Goals: Dual Scoring for Parameter and Data Selection in LLM Fine-Tuning

Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences(中国科学院大学杭州高等研究院);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学院)

Xinrui Chen, Liu Yang, Ou Wu

AI总结:

本文提出DualSFT算法,通过共享梯度统计生成参数掩码和数据子集,提升微调性能与稳定性-可塑性平衡。

摘要:

在大型语言模型(LLM)微调中,参数和数据选择是减少微调成本的常见策略,但通常由独立的评分机制驱动。当参数掩码和数据子集共同决定受限微调时,这种分离导致冗余开销并使协调选择困难。本文将参数和数据选择视为在共同验证目标下的两个双层选择问题,并推导出共享的局部响应-替代评分规则。在第一和第二阶验证改进近似下,参数重要性和数据效用作为单个梯度交互矩阵的列和行聚合,产生闭合形式的行-列对应关系,以共同提取两种信号。基于此结构,我们提出了DualSFT(双选择微调),一种单次双评分算法,从共享梯度统计中生成参数掩码和数据子集。在3B-9B LLMs上,单轴DualSFT变体在比较组内增强目标任务性能和稳定性-可塑性平衡,而完整DualSFT在匹配预算下比顺序混合基线产生更有利的联合约束平衡。

英文摘要:

In Large Language Model (LLM) fine-tuning, parameter and data selection are common strategies for reducing fine-tuning cost, yet they are typically driven by separate scoring mechanisms. When a parameter mask and data subset jointly determine restricted fine-tuning, this separation incurs redundant overhead and makes coordinated selection difficult. We cast parameter and data selection as two bilevel selection problems under a common validation objective and derive a shared local response-surrogate scoring rule. Under first- and second-order validation-improvement approximations, parameter importance and data utility emerge as column-wise and row-wise aggregations of a single gradient interaction matrix, yielding a closed-form row-column correspondence for co-extracting both signals. Building on this structure, we propose DualSFT (Dual-Selection Fine-Tuning), a one-shot dual-scoring algorithm that produces a parameter mask and data subset from shared gradient statistics. On 3B-9B LLMs, single-axis DualSFT variants strengthen target-task performance and stability-plasticity trade-offs within their comparison groups, while full DualSFT yields a more favorable joint-constrained trade-off than sequential hybrid baselines under matched budgets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑