PRISM:通过各向异性频谱塑造实现结构化优化
PRISM: Structured Optimization via Anisotropic Spectral Shaping
AI总结:
PRISM通过各向异性频谱塑造技术,结合部分第二阶信息提升第一阶优化方法,实现高效结构化优化。
摘要:
我们提出了PRISM,一种增强第一阶频谱下降方法(如Muon)的优化器,利用部分第二阶信息。它通过创新增强的极分解构建高效的低秩准第二阶预条件器。这种机制使PRISM能够实现各向异性频谱塑造,即自适应地抑制高方差子空间的更新,同时在信号主导方向保持更新强度。关键在于,这在计算开销和内存使用上都比第一阶基线方案最小化。PRISM展示了将曲率自适应特性整合到频谱优化范式中的实用策略。
英文摘要:
We propose PRISM, an optimizer that enhances first-order spectral descent methods like Muon with partial second-order information. It constructs an efficient, low-rank quasi-second-order preconditioner via innovation-augmented polar decomposition. This mechanism enables PRISM to perform anisotropic spectral shaping, which adaptively suppresses updates in high-variance subspaces while preserving update strength in signal-dominated directions. Crucially, this is achieved with minimal computational overhead and zero additional memory compared to first-order baselines. PRISM demonstrates a practical strategy for integrating curvature-adaptive properties into the spectral optimization paradigm.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。