arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.06583cs.LGcs.AI
收藏

用于神经网络训练的自适应多重最优学习因子

Adaptive multiple optimal learning factors for neural network training

The University of Texas at Arlington(德克萨斯大学阿灵顿分校)

Jeshwanth Challagundla

AI总结:

该研究提出 AMOLF 算法,依据每次乘法的误差变化自适应调整学习因子数量,并结合权重曲率分组与 Hessian 压缩技术,在实验中优于 OWO-MOLF 和 Levenberg-Marquardt。

摘要:

本论文提出了一种神经网络训练的新方法,以解决确定最优学习因子数量这一挑战。所提出的自适应多重最优学习因子(Adaptive Multiple Optimal Learning Factors,AMOLF)算法根据每次乘法运算的误差变化动态调整学习因子数量,从而提升训练效率和准确率。论文还介绍了基于目标函数曲率对权重进行分组的技术,以及压缩大型 Hessian 矩阵的技术。实验结果表明,与 OWO-MOLF 和 Levenberg-Marquardt 等现有方法相比,AMOLF 具有更优越的性能。

英文摘要:

This thesis presents a novel approach to neural network training that addresses the challenge of determining the optimal number of learning factors. The proposed Adaptive Multiple Optimal Learning Factors (AMOLF) algorithm dynamically adjusts the number of learning factors based on the error change per multiply, leading to improved training efficiency and accuracy. The thesis also introduces techniques for grouping weights based on the curvature of the objective function and for compressing large Hessian matrices. Experimental results demonstrate the superior performance of AMOLF compared to existing methods like OWO-MOLF and Levenberg-Marquardt.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑