arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.07198quant-ph
收藏

正交神经网络的经典和量子算法

Classical and Quantum Algorithms for Orthogonal Neural Networks

Iordanis Kerenidis, Jonas Landman, Natansh Mathur

摘要:

正交神经网络最近被引入作为一种新的类型的神经网络强加正交的权重矩阵。它们可以实现更高的精度,并避免深层架构的渐逝或爆炸性梯度。已经提出了几种经典的梯度下降方法来保持正交性,同时更新的权重矩阵,但这些技术的运行时间长,或只提供近似正交。在本文中,我们介绍了一种新型的神经网络层称为金字塔电路,它实现了正交矩阵乘法。它允许具有完美正交性的梯度下降,具有与标准层相同的渐进运行时间。该算法受到量子计算的启发,因此可以应用于经典计算机以及近期的量子计算机。它可以成为量子神经网络和更快的正交神经网络的构建模块。

英文摘要:

Orthogonal neural networks have recently been introduced as a new type of neural networks imposing orthogonality on the weight matrices. They could achieve higher accuracy and avoid evanescent or explosive gradients for deep architectures. Several classical gradient descent methods have been proposed to preserve orthogonality while updating the weight matrices, but these techniques suffer from long running times or provide only approximate orthogonality. In this paper, we introduce a new type of neural network layer called Pyramidal Circuit, which implements an orthogonal matrix multiplication. It allows for gradient descent with perfect orthogonality with the same asymptotic running time as a standard layer. This algorithm is inspired by quantum computing and can therefore be applied on a classical computer as well as on a near term quantum computer. It could become the building block for quantum neural networks and faster orthogonal neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑