arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.08620cs.CLcs.AIcs.LG
收藏

学得更多,忘记得更少:面向大语言模型的梯度感知数据选择方法

Learn More, Forget Less: A Gradient-Aware Data Selection Approach for LLM

Fu Foundation School of Engineering and Applied Science, Columbia University(哥伦比亚大学工程与应用科学学院);School of Computer Science and Engineering, Beihang University(北航计算机科学与工程学院)

Yibai Liu, Shihang Wang, Zeming Liu, Zheming Song, Junzhe Wang, Jingjing Liu, Qingjie Liu, Yunhong Wang

AI总结:

针对大语言模型领域监督微调(SFT)资源密集且易因灾难性遗忘(CF)损失通用能力的问题,提出自适应性梯度感知数据选择方法GrADS,通过初步训练梯度选有效样本,实验显示仅用5%所选数据性能超全量微调,50%数据提升显著且缓解CF。

摘要:

尽管大语言模型(LLMs)在众多任务中取得了令人瞩目的成就,但监督微调(SFT)对于使这些模型适应特定领域仍然至关重要。然而,用于领域专业化的SFT可能资源密集,有时还会因灾难性遗忘(CF)导致通用能力的性能下降。为解决这些问题,我们提出一种自适应梯度感知数据选择方法(GrADS),用于LLMs的监督微调,该方法通过分析初步训练阶段获得的梯度来识别有效训练数据子集。具体而言,我们设计了自引导标准,利用梯度的幅度和统计分布来优先处理对模型学习过程贡献最大的样本。这种方法能够获取代表性样本,增强LLMs对特定领域任务的理解。通过在医学、法律和金融等不同领域对各种LLMs进行广泛实验,GrADS展现出显著的效率和成本效益。值得注意的是,仅使用5%的GrADS选择数据,LLMs的性能就已超过在整个数据集上微调的模型,而使用50%的数据则带来显著提升,同时灾难性遗忘得到大幅缓解。我们将在后续发布GrADS的代码。

英文摘要:

Despite large language models (LLMs) have achieved impressive achievements across numerous tasks, supervised fine-tuning (SFT) remains essential for adapting these models to specialized domains. However, SFT for domain specialization can be resource-intensive and sometimes leads to a deterioration in performance over general capabilities due to catastrophic forgetting (CF). To address these issues, we propose a self-adaptive gradient-aware data selection approach (GrADS) for supervised fine-tuning of LLMs, which identifies effective subsets of training data by analyzing gradients obtained from a preliminary training phase. Specifically, we design self-guided criteria that leverage the magnitude and statistical distribution of gradients to prioritize examples that contribute the most to the model's learning process. This approach enables the acquisition of representative samples that enhance LLMs understanding of domain-specific tasks. Through extensive experimentation with various LLMs across diverse domains such as medicine, law, and finance, GrADS has demonstrated significant efficiency and cost-effectiveness. Remarkably, utilizing merely 5% of the selected GrADS data, LLMs already surpass the performance of those fine-tuned on the entire dataset, and increasing to 50% of the data results in significant improvements! With catastrophic forgetting substantially mitigated simultaneously. We will release our code for GrADS later.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑