崩溃、修复与漂移：剖析语音语言模型中的微调动态
Collapse, Repair, and Drift: Dissecting Fine Tuning Dynamics in Speech Language Models
AI总结：
该研究针对Qwen3-ASR等语音语言模型，通过梯度频谱分析等方法剖析其监督微调的崩溃、修复与漂移动态，发现瞬态崩溃并非源域性能下降的必要条件。
摘要：
监督微调（SFT）通常通过训练损失和评估损失进行监控，但损失无法完全表征模型功能在同一轨迹上的演变。我们通过Qwen3-ASR、MiMo-Audio和FireRed-LLM的方言适应研究这一差距，学习率范围为$10^{-5}$至$10^{-3}$。字符错误率（CER）显示出明显的瞬态崩溃与恢复，其严重程度和时间结构仅在很小程度上被损失反映。双向层修补揭示了可恢复崩溃期间存在强烈的跨检查点功能不兼容性，且具有依赖模型的深度分布，在恢复过程中会重新组织。仅成对梯度相似性无法可靠指示优化稳定性，而原始梯度频谱分析能更清晰地区分稳定与不稳定优化：成功收敛的运行呈现出宽泛且时间稳定的频谱能量分布，而失败的运行则保持频谱集中或表现出持续波动。最后，AdamW更新转移可绕过明显的早期崩溃，但训练后期仍会出现显著的普通话性能下降。这些结果揭示了三种可分离的SFT动态：崩溃、修复与漂移，且瞬态崩溃并非源域持续性能下降的必要条件。
英文摘要：
Supervised fine tuning (SFT) is commonly monitored through training and evaluation loss, but loss does not fully characterize how model functionality evolves along the same trajectory. We study this gap through dialect adaptation of Qwen3-ASR, MiMo-Audio, and FireRed-LLM across learning rates from $10^{-5}$ to $10^{-3}$. Character error rate (CER) reveals pronounced transient collapse and recovery whose severity and temporal structure are only weakly reflected by loss. Bidirectional layer patching reveals strong cross checkpoint functional incompatibility during recoverable collapse, with model dependent depth profiles that reorganize during recovery. Pairwise gradient similarity alone does not reliably indicate optimization stability, whereas raw gradient spectral analysis provides a clearer structural distinction between stable and unstable optimization: successfully converged runs approach a broad and temporally stable spectral energy distribution, while failed runs remain spectrally concentrated or exhibit persistent fluctuations. Finally, AdamW update transfer bypasses pronounced early collapse, yet substantial Mandarin degradation still emerges later in training. These results motivate three separable SFT dynamics: collapse, repair, and drift, and show that transient collapse is not necessary for persistent source domain degradation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
