基于梯度正交性的训练数据选择用于高效的领域适应
Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation
AI总结：
本文提出OGS方法，通过梯度正交性选择训练数据，提升领域适应的效率和性能。
AI中文摘要：
在为专门领域微调大型语言模型（LLMs）时，往往需要在获取领域专业知识和保留一般推理能力之间做出权衡，这种现象称为灾难性遗忘。现有解决方案面临二元对立：梯度手术方法提供几何安全性，但通过在线投影带来巨大的计算成本；而高效数据选择方法减少开销，但对引起冲突的梯度方向缺乏感知。在本文中，我们提出了正交梯度选择（OGS），一种以数据为中心的方法，旨在调和领域性能、一般能力保留和训练效率。OGS将梯度投影的几何洞察从优化器转移到数据选择阶段，通过将数据选择视为一个受约束的决策过程来实现。利用轻量级的导航模型和强化学习技术，OGS动态地识别出梯度与一般知识锚点正交的训练样本。这种方法确保了目标模型的自然安全更新，而无需修改优化器或产生运行时投影成本。在医疗、法律和金融领域进行的实验表明，OGS取得了优异的结果，显著提高了领域性能和训练效率，同时在如GSM8K等一般任务上保持或甚至提升了性能。
英文摘要：
Fine-tuning large language models (LLMs) for specialized domains often necessitates a trade-off between acquiring domain expertise and retaining general reasoning capabilities, a phenomenon known as catastrophic forgetting. Existing remedies face a dichotomy: gradient surgery methods offer geometric safety but incur prohibitive computational costs via online projections, while efficient data selection approaches reduce overhead but remain blind to conflict-inducing gradient directions. In this paper, we propose Orthogonal Gradient Selection (OGS), a data-centric method that harmonizes domain performance, general capability retention, and training efficiency. OGS shifts the geometric insights of gradient projection from the optimizer to the data selection stage by treating data selection as a constrained decision-making process. By leveraging a lightweight Navigator model and reinforcement learning techniques, OGS dynamically identifies training samples whose gradients are orthogonal to a general-knowledge anchor. This approach ensures naturally safe updates for target models without modifying the optimizer or incurring runtime projection costs. Experiments across medical, legal, and financial domains demonstrate that OGS achieves excellent results, significantly improving domain performance and training efficiency while maintaining or even enhancing performance on general tasks such as GSM8K.
