SUDA-Muon：完全去中心化穆恩的结构设计原则和边界
SUDA-Muon: Structural Design Principles and Boundaries for Fully Decentralized Muon
AI总结：
本文提出SUDA-Muon算法，通过统一的对偶通信模板实现算法模块化，证明了非渐近收敛性，并展示了在非中心化设置下的有效性。
摘要：
完全去中心化穆恩难以实现，因其非线性矩阵符号运算不与线性 gossip 平均运算可交换。这使得去中心化穆恩成为一个结构设计问题：在设计算法时，必须区分模块化组件与非模块化组件。我们提出SUDA-Muon，通过统一的对偶通信模板SUDA实现这种分离；在此模板中，ED/D$^2$、EXTRA和梯度跟踪成为模块化核心选择。我们证明了在核范数几何中拓扑分离的非渐近收敛保证：主导项规模为$\mathcal{O}((1+σ/\sqrt{N})K^{-1/4})$，不显式涉及图量，将通信骨干识别为结构设计中的模块轴。然后我们建立了两个互补的非模块化边界。内部，跟踪在前极化是必要的，以避免在异质目标下出现非静态固定点。外部，在没有中央服务器的情况下，完全去中心化方法无法执行联邦平均后再极化更新；我们显示，这种非模块化本地极化后再平均设计是导致无法表现出线性加速的根本原因。在CIFAR-100和GPT-2微调实验中支持相同的图景：统一的模板使不同的通信算法直接可比。在温和的近独立同分布（IID）情况下，所得到的变种表现相似，而在更困难的长时间非IID CIFAR-100设置中，SUDA-Muon比DeMuon实现更高的准确性和更低的损失。
英文摘要：
Fully decentralized Muon is difficult because its nonlinear matrix-sign operator does not commute with linear gossip averaging. This makes decentralized Muon a structural design problem: in designing the algorithm, one must distinguish modular components from non-modular ones. We propose \sudamuon{}, which realizes this separation through a unified primal--dual communication template called SUDA; within this template, ED/D$^2$, EXTRA, and gradient tracking become modular backbone choices. We prove a topology-separated non-asymptotic convergence guarantee in the nuclear-norm geometry: the dominant term scales as $\mathcal{O}((1+σ/\sqrt{N})K^{-1/4})$ and does not explicitly involve graph quantities, identifying the communication backbone as the modular axis in the structure design. We then establish two complementary non-modular boundaries. Internally, tracking-before-polarization is necessary for this natural no-tracking variant to avoid non-stationary fixed points under heterogeneous objectives. Externally, in the absence of a central server, a fully decentralized method cannot perform the federated average-then-polarize update; we show that this non-modular local-polarize-then-average design is the essential reason why can fail to exhibit linear speedup. Experiments on CIFAR-100 and GPT-2 fine-tuning support the same picture: the unified template makes different communication algorithms directly comparable. In mild near-IID regimes, the resulting variants perform similarly, while in the more difficult long-horizon non-IID CIFAR-100 setting, \sudamuon{} achieves higher accuracy and lower loss than \textsc{DeMuon}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
