arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.01730cs.CV
收藏

基于分支定界的深度神经网络训练

Training Deep Neural Networks via Branch-and-Bound

University of Kansas(堪萨斯大学);Worcester Polytechnic Institute(伍斯特理工学院);Ryerson University(瑞尔森大学)

Yuanwei Wu, Ziming Zhang, Guanghui Wang

摘要:

在本文中,我们提出了BPGrad,一种新的深度神经网络训练的近似算法,基于通过分支定界的可行域的自适应估计。该方法基于目标函数的Lipschitz连续性假设,因此,它可以自适应地确定当前梯度的步长给定以前的更新的历史。我们证明,通过重复这样的分支和修剪过程,它可以在有限的迭代达到最优解。提出了一种基于BPGrad的计算效率高的求解器来训练深度神经网络。实验结果表明,BPGrad求解器在实践中效果良好,并且在对象识别、检测和分割任务中与其他随机优化方法相比具有优势。代码可在\url{https://github.com/RyanCV/BPGrad}上获得。

英文摘要:

In this paper, we propose BPGrad, a novel approximate algorithm for deep nueral network training, based on adaptive estimates of feasible region via branch-and-bound. The method is based on the assumption of Lipschitz continuity in objective function, and as a result, it can adaptively determine the step size for the current gradient given the history of previous updates. We prove that, by repeating such a branch-and-pruning procedure, it can achieve the optimal solution within finite iterations. A computationally efficient solver based on BPGrad has been proposed to train the deep neural networks. Empirical results demonstrate that BPGrad solver works well in practice and compares favorably to other stochastic optimization methods in the tasks of object recognition, detection, and segmentation. The code is available at \url{https://github.com/RyanCV/BPGrad}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑