发表机构
University of Macau; Alibaba Group; Nanjing University; KAUST; Wuhan University; Xiamen University（澳门大学; 阿里巴巴集团; 南京大学; 阿卜杜拉国王科技大学; 武汉大学; 厦门大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对全参数微调中的参数冗余、数据质量不一致和灾难性遗忘问题，提出神经元引导的微调框架，通过三种协同机制统一微调生命周期，在效率和性能上超越现有方法。
AI 中文摘要
现有的监督微调范式，特别是全参数微调，常常受到参数冗余、数据质量不一致和灾难性遗忘的困扰，而当前的方法通常孤立地解决这些问题，缺乏一个统一的优化信号来桥接数据选择、参数更新和知识保留。为了解决这一问题，我们提出了神经元引导的微调（NGFT），这是一个整体框架，利用神经元激活模式作为通用代理来统一微调生命周期。NGFT通过三种协同机制运作：（1）自适应任务特定神经元选择，在单次前向传播中识别关键神经元，以集中更新并减少冗余；（2）基于激活的数据选择，优先选择信息密集的样本，以最大化对关键神经元的贡献；（3）神经元激活对齐，一种新颖的损失函数，将激活锚定到预训练状态，深化表示学习并保留通用知识。在三个模型上的实验结果表明，在领域特定和通用基准上，NGFT在效率和性能方面均显著优于现有的主流微调方法，同时有效缓解了灾难性遗忘。
英文摘要
Existing Supervised Fine-Tuning paradigms, particularly Full Parameter Fine-Tuning are often plagued by parameter redundancy, inconsistent data quality, and catastrophic forgetting, which current methods typically address in isolation and lack a unified optimization signal to bridge data selection, parameter updates, and knowledge preservation. To address this, we propose Neuron-Guided Fine-Tuning (NGFT), a holistic framework that leverages neuron activation patterns as a universal proxy to unify the fine-tuning lifecycle. NGFT operates via three synergistic mechanisms: (1) Adaptive Task-Specific Neuron Selection, which identifies essential neurons in a single forward pass to concentrate updates and reduce redundancy; (2) Activation-Based Data Selection, which prioritizes information-dense samples that maximize contribution to key neurons; and (3) Neuron Activation Alignment, a novel loss function that anchors activations to pre-trained states, deepening representation learning and preserving general knowledge. Experimental results across three models across both domain-specific and general benchmarks demonstrate that NGFT significantly outperforms existing mainstream fine-tuning methods in both efficiency and performance, while effectively mitigating catastrophic forgetting.
CommentsEMNLP 2026 Findings. Codes are available at: https://github.com/NLP2CT/NGFT
