arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.15399cs.LG
收藏

利用参数对称性改进学习优化

Improving Learning to Optimize Using Parameter Symmetries

University of Wisconsin-Madison(威斯康星大学麦迪逊分校);University of California, San Diego(加州大学圣地亚哥分校)

Guy Zamir, Aryan Dokania, Bo Zhao, Rose Yu

AI总结:

本文分析并验证了利用参数空间对称性的学习优化算法,理论上证明其局部近似牛顿法,并通过基准测试和动量增强展示了性能提升,凸显了对称性在元优化中的潜力。

摘要:

我们分析了一种利用参数空间对称性来提高优化效率的学习优化(L2O)算法。先前的研究表明,联合学习对称变换和局部更新能提升元优化器的性能。与此一致,我们的理论分析证明,即使未识别出最优群元素,该方法在局部上近似于牛顿法。我们进一步提供了一个示例,其中算法在训练过程中可证明地学习到正确的对称变换。为了实证评估带有传送的L2O,我们引入了一个基准,分析了其成功与失败案例,并表明诸如动量等增强手段能进一步提升性能。我们的结果凸显了利用神经网络参数空间对称性来推进元优化的潜力。

英文摘要:

We analyze a learning-to-optimize (L2O) algorithm that exploits parameter space symmetry to enhance optimization efficiency. Prior work has shown that jointly learning symmetry transformations and local updates improves meta-optimizer performance. Supporting this, our theoretical analysis demonstrates that even without identifying the optimal group element, the method locally resembles Newton's method. We further provide an example where the algorithm provably learns the correct symmetry transformation during training. To empirically evaluate L2O with teleportation, we introduce a benchmark, analyze its success and failure cases, and show that enhancements like momentum further improve performance. Our results highlight the potential of leveraging neural network parameter space symmetry to advance meta-optimization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑