数据并行全微调中的沉默不一致:诊断工作级别优化不一致
Silent Inconsistency in Data-Parallel Full Fine-Tuning: Diagnosing Worker-Level Optimization Misalignment
AI总结:
本文提出了一种轻量诊断框架,用于检测数据并行微调中工作级别优化的不一致问题,通过损失分散度、梯度范数分散度和方向一致性指标,揭示隐藏的不稳定性模式。
摘要:
数据并行(DP)训练采用同步all-reduce机制是大语言模型(LLMs)全参数微调的主要范式。尽管参数同步保证了每次迭代后模型权重的数值等价性,但并不必然意味着在梯度聚合前工作级别优化动态的对齐。本文识别并研究了这种隐含的不匹配,称为"沉默不一致",其中跨工作节点的损失和梯度差异可能在传统聚合监控信号下保持不可见。我们提出了一种轻量、模型无关的诊断框架,利用标准流程中 readily available 的训练信号量化工作级别的一致性。具体而言,我们引入了三个互补的指标:损失分散度、梯度范数分散度以及通过跨工作节点余弦相似性测量的梯度方向一致性。所提出的指标具有极小的开销,且不需修改模型架构、同步机制或优化算法。我们通过在8-NPU DP设置下,对1B参数的openPangu-Embedded-1B-V1.1模型在tatsu-lab/alpaca数据集上进行全微调,验证了该框架。在受控的跨秩随机性扰动下,实验结果表明,逐步去同步的数据洗牌和随机种子导致损失/梯度分散度显著增加,方向对齐性降低,尽管全局平均损失曲线平滑。这些发现表明,所提出的指标能够提供对大规模DP微调中隐藏不稳定性模式的可操作可见性,从而实现更可靠的诊断和配置评估。
英文摘要:
Data-parallel (DP) training with synchronous all-reduce is a dominant paradigm for full-parameter fine-tuning of large language models (LLMs). While parameter synchronization guarantees numerical equivalence of model weights after each iteration, it does not necessarily imply alignment of worker-level optimization dynamics before gradient aggregation. This paper identifies and studies this latent mismatch, termed \emph{silent inconsistency}, where cross-worker divergence in losses and gradients can remain invisible under conventional aggregated monitoring signals. We propose a lightweight, model-agnostic diagnostic framework that quantifies worker-level consistency using training signals readily available in standard pipelines. Specifically, we introduce three complementary metrics: loss dispersion, gradient-norm dispersion, and gradient-direction consistency measured by inter-worker cosine similarity. The proposed metrics incur negligible overhead and require no modification to model architecture, synchronization mechanisms, or optimization algorithms. We validate the framework by fully fine-tuning the 1B-parameter \texttt{openPangu-Embedded-1B-V1.1} model on the \texttt{tatsu-lab/alpaca} dataset using an 8-NPU DP setup, under controlled perturbations of cross-rank stochasticity. Experimental results show that progressively desynchronized data shuffling and random seeds lead to substantial increases in loss/gradient dispersion and reduced directional alignment, despite smooth globally averaged loss curves. These findings demonstrate that the proposed indicators provide actionable visibility into hidden instability modes in large-scale DP fine-tuning, enabling more reliable diagnosis and configuration assessment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。