arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.01401cs.LGcs.AI
收藏

基于Newton方法的卷积神经网络并行处理

Newton methods based convolution neural networks using parallel processing

Panjab University(旁遮普大学)

Ujjwal Thakur, Anuj Sharma

AI总结:

针对卷积神经网络训练中二阶Newton方法计算Hessian矩阵复杂且耗时的问题,本文使用完整数据并在小批量计算中采用并行处理,显著缩短了训练时间。

摘要:

卷积神经网络的训练是一个高维且非凸的优化问题。目前,在无法可靠设置参数化学习率的情况下,其效率低下。一些过去的工作引入了Newton方法来训练深度神经网络。用于卷积神经网络的Newton方法涉及复杂的操作。在二阶方法中寻找Hessian矩阵变得非常复杂,因为我们主要对图像数据使用有限差分法。用于卷积神经网络的Newton方法通过使用子采样Hessian Newton方法来处理此问题。在本文中,我们使用了完整数据,而不是一次仅处理部分数据的子采样方法。此外,我们在小批量计算中使用了并行处理而不是串行处理。本研究中使用并行处理获得的结果,在耗时上优于先前的方法。

英文摘要:

Training of convolutional neural networks is a high dimensional and a non-convex optimization problem. At present, it is inefficient in situations where parametric learning rates can not be confidently set. Some past works have introduced Newton methods for training deep neural networks. Newton methods for convolutional neural networks involve complicated operations. Finding the Hessian matrix in second-order methods becomes very complex as we mainly use the finite differences method with the image data. Newton methods for convolutional neural networks deals with this by using the sub-sampled Hessian Newton methods. In this paper, we have used the complete data instead of the sub-sampled methods that only handle partial data at a time. Further, we have used parallel processing instead of serial processing in mini-batch computations. The results obtained using parallel processing in this study, outperform the time taken by the previous approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑