arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.15419cs.LGcs.NAmath.NA
收藏

矩阵与线性映射的Frobenius型范数和内积及其在神经网络训练中的应用

Frobenius-Type Norms and Inner Products of Matrices and Linear Maps with Applications to Neural Network Training

Heidelberg University(海德堡大学);University of Bayreuth(拜罗伊特大学)

Roland Herzog, Frederik Köhne, Leonie Kreis, Anton Schiela

AI总结:

本文将经典Frobenius范数推广为一族依赖定义域和陪域内积的Frobenius型范数,并利用由此获得的额外自由度对神经网络训练进行预条件处理。

摘要:

Frobenius范数是矩阵范数中的常用选择。特别是,底层的Frobenius内积通常用于评估目标函数关于矩阵变量的梯度,例如在神经网络训练中出现的矩阵变量。我们对线性映射或矩阵的Frobenius范数和内积提供了一个更广泛的视角,并建立了它们对定义域和陪域空间中内积的依赖性。这表明经典的Frobenius范数只是一族更一般的Frobenius型范数中的一个特殊元素。这一认识所带来的显著额外自由度可以用于(除其他用途外)对神经网络训练进行预条件处理。

英文摘要:

The Frobenius norm is a frequent choice of norm for matrices. In particular, the underlying Frobenius inner product is typically used to evaluate the gradient of an objective with respect to matrix variable, such as those occuring in the training of neural networks. We provide a broader view on the Frobenius norm and inner product for linear maps or matrices, and establish their dependence on inner products in the domain and co-domain spaces. This shows that the classical Frobenius norm is merely one special element of a family of more general Frobenius-type norms. The significant extra freedom furnished by this realization can be used, among other things, to precondition neural network training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑