arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1601.07227math.NAcs.NAcs.NE
收藏

学习Strassen乘法的网络

A network that learns Strassen multiplication

Cornell University(康奈尔大学)

Veit Elser

AI总结:

本文研究仅含乘法非线性组件的神经网络,通过限制乘法数量迫使网络发现矩阵乘法规则,采用保守学习规则实现低秩分解。

摘要:

我们研究仅含乘法非线性组件的神经网络,以在数据精确表示至关重要的上下文中测试新的训练规则。这些网络需在多个示例中发现矩阵乘法规则。通过限制乘法数量,网络被迫发现Strassen乘法规则。这是寻找n×n矩阵乘法张量M_n低秩分解的数学等价过程。我们用保守学习规则训练这些网络,该规则在接收到输入-输出对时对权重做出最小调整以获得正确输出。保守学习需要数千个示例才能找到M_2的秩7分解,以及10^5个示例才能找到M_3的秩23分解(目前已知的最低)。高精度至关重要,尤其是对于M_3,以区分真正的分解和“边界近似”。

英文摘要:

We study neural networks whose only non-linear components are multipliers, to test a new training rule in a context where the precise representation of data is paramount. These networks are challenged to discover the rules of matrix multiplication, given many examples. By limiting the number of multipliers, the network is forced to discover the Strassen multiplication rules. This is the mathematical equivalent of finding low rank decompositions of the $n\times n$ matrix multiplication tensor, $M_n$. We train these networks with the conservative learning rule, which makes minimal changes to the weights so as to give the correct output for each input at the time the input-output pair is received. Conservative learning needs a few thousand examples to find the rank 7 decomposition of $M_2$, and $10^5$ for the rank 23 decomposition of $M_3$ (the lowest known). High precision is critical, especially for $M_3$, to discriminate between true decompositions and "border approximations".

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑