arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.02366cs.LGmath.OCstat.ML
收藏

深度神经网络中Hessian函数的负特征值

Negative eigenvalues of the Hessian in deep neural networks

Mila, University of Montreal(米拉-蒙特利尔大学);Google Brain(谷歌大脑)

Guillaume Alain, Nicolas Le Roux, Pierre-Antoine Manzagol

摘要:

深度网络的损失函数已知是非凸的,但这种非凸性的确切性质仍然是一个活跃的研究领域。在这项工作中,我们通过Hessian矩阵的特征分解来研究深度网络的损失景观。特别是,我们研究了负特征值的重要性,以及在适当处理它们时可以观察到的好处。

英文摘要:

The loss function of deep networks is known to be non-convex but the precise nature of this nonconvexity is still an active area of research. In this work, we study the loss landscape of deep networks through the eigendecompositions of their Hessian matrix. In particular, we examine how important the negative eigenvalues are and the benefits one can observe in handling them appropriately.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑