发表机构
Institut de Recherche Technologique Saint-Exupery; IRIT; SNCF（圣埃克苏佩里技术研究所; 图卢兹计算机科学研究所; 法国国家铁路公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究 Muon 优化器在卷积上的理论不匹配，提出卷积牛顿-舒尔茨方法，实验表明两者精度相当，但理论对齐可能过度约束更新，凸显 Muon 实际有效性。
AI 中文摘要
Muon 是一种以高效著称的优化器，其对矩阵值更新有清晰的解释，但卷积核存储为四维张量。标准实现将这些张量重塑为矩阵，这种捷径破坏了 Muon 背后的理论理解。为探究此问题，我们直接在卷积算子几何中形式化相应的优化目标，并引入卷积牛顿-舒尔茨（Conv-NS），该方法在该几何中近似极因子，同时保留核支持。在快速训练实验中，Conv-NS 和基于重塑的 Muon 均计算高效，并在 CIFAR-10 和 ImageNet 分类任务上达到相当精度。然而，由于人们可能预期理论对齐的 Conv-NS 优于基于重塑的 Muon，我们研究了实践与理论理解之间的这种不匹配，假设精确的卷积正交化可能过度约束更新。这些发现凸显了 Muon 强大的实际性能，同时为其在卷积上的进一步发展开辟了方向。我们的代码公开于 \href{ this https URL }{github conv-muon}。
英文摘要
Muon, an optimizer known for its efficiency, has a clear interpretation for matrix-valued updates, but convolutional kernels are stored as four-dimensional tensors. Standard implementations reshape these tensors into matrices, a shortcut which breaks the theoretical understanding behind Muon. To investigate this, we formalize the corresponding optimization objective directly in convolutional operator geometry and introduce Convolutional Newton-Schulz (Conv-NS), which approximates the polar factor in this geometry while preserving kernel support. When applied in fast training experiments, Conv-NS and reshape-based Muon are both computationally efficient and achieve comparable accuracy on CIFAR-10 and ImageNet classification tasks. However, as one could expect a theoretically aligned Conv-NS to outperform reshape-based Muon, we investigate this mismatch between practice and theoretical understanding, with the hypothesis that exact convolutional orthogonalization may overconstrain updates. These findings highlight Muon's strong practical performance while opening directions for its further development on convolutions. Our code is publicly available at \href{https://github.com/thib-s/muonconv-cifar10-airbench}{github conv-muon}.
