arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.01898cs.LGcs.CL
收藏

带Transformers的线性代数

Linear algebra with transformers

Meta AI

François Charton

摘要:

Transformers只能从示例中学习执行数值计算。我研究了线性代数的九个问题,从基本的矩阵运算到特征值分解和求逆,并介绍和讨论了表示实数的四种编码方案。在所有问题上,在随机矩阵集上训练的Transformers都达到了高精度(超过90%)。该模型对噪声具有鲁棒性,并且可以推广到其训练分布之外。特别是,训练来预测拉普拉斯分布特征值的模型可以推广到不同类型的矩阵:维格纳矩阵或具有正特征值的矩阵。反之则不然。

英文摘要:

Transformers can learn to perform numerical computations from examples only. I study nine problems of linear algebra, from basic matrix operations to eigenvalue decomposition and inversion, and introduce and discuss four encoding schemes to represent real numbers. On all problems, transformers trained on sets of random matrices achieve high accuracies (over 90%). The models are robust to noise, and can generalize out of their training distribution. In particular, models trained to predict Laplace-distributed eigenvalues generalize to different classes of matrices: Wigner matrices or matrices with positive eigenvalues. The reverse is not true.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑