arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2201.11137cs.LGastro-ph.COhep-thmath.OCstat.ML
收藏

Born-Infeld (BI) 用于人工智能:能量守恒下降(ECD)优化方法

Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimization

Stanford University(斯坦福大学)

G. Bruno De Luca, Eva Silverstein

AI总结:

提出基于能量守恒哈密顿动力学的Born-Infeld优化框架,利用混沌特性自然减速,避免高局部极小值,在机器学习与PDE求解中表现优于GD+momentum。

摘要:

我们提出了一种基于强混合(混沌)区域中能量守恒哈密顿动力学的新型优化框架,并在解析和数值上建立了其关键性质。原型是 Born-Infeld 动力学的一种离散化,其平方相对论速度极限依赖于目标函数。这类无摩擦、能量守恒的优化器在接近最小损失(该损失主导系统的相空间体积)之前不受阻碍地运行,并在最小损失附近自然减速。基于对混沌系统(如动力学台球)的研究,我们制定了一种特定算法,在机器学习、偏微分方程求解任务(包括泛化)上表现出良好性能。该算法无法停留在高局部极小值,这是非凸损失函数中的一个优势,并且在浅谷中比 GD+momentum 更快。

英文摘要:

We introduce a novel framework for optimization based on energy-conserving Hamiltonian dynamics in a strongly mixing (chaotic) regime and establish its key properties analytically and numerically. The prototype is a discretization of Born-Infeld dynamics, with a squared relativistic speed limit depending on the objective function. This class of frictionless, energy-conserving optimizers proceeds unobstructed until slowing naturally near the minimal loss, which dominates the phase space volume of the system. Building from studies of chaotic systems such as dynamical billiards, we formulate a specific algorithm with good performance on machine learning and PDE-solving tasks, including generalization. It cannot stop at a high local minimum, an advantage in non-convex loss functions, and proceeds faster than GD+momentum in shallow valleys.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑