arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.07085cs.LGcs.NEstat.ML
收藏

神经状态寄存器

Neural Status Registers

ETH Zurich(苏黎世联邦理工学院)

Lukas Faber, Roger Wattenhofer

摘要:

标准的神经网络可以学习数学运算,但不能外推。外推意味着该模型可以应用于更大的数字,远远超出训练期间观察到的数字。最近的架构处理算术运算,并可以推断;然而,同样重要的定量推理问题仍然没有解决。在这项工作中,我们提出了一种新的建筑元素,神经状态寄存器(NSR),定量推理的数字。我们的NSR将物理状态寄存器的离散位逻辑放宽为连续数字,并允许使用梯度下降进行端到端学习。实验表明,NSR实现的解决方案外推到比训练集中的数字大许多数量级的数字。我们成功地训练了NSR的数字比较,分段不连续函数,在序列中计数,递归地找到最小值,在图中找到最短路径,并比较图像中的数字。

英文摘要:

Standard Neural Networks can learn mathematical operations, but they do not extrapolate. Extrapolation means that the model can apply to larger numbers, well beyond those observed during training. Recent architectures tackle arithmetic operations and can extrapolate; however, the equally important problem of quantitative reasoning remains unaddressed. In this work, we propose a novel architectural element, the Neural Status Register (NSR), for quantitative reasoning over numbers. Our NSR relaxes the discrete bit logic of physical status registers to continuous numbers and allows end-to-end learning with gradient descent. Experiments show that the NSR achieves solutions that extrapolate to numbers many orders of magnitude larger than those in the training set. We successfully train the NSR on number comparisons, piecewise discontinuous functions, counting in sequences, recurrently finding minimums, finding shortest paths in graphs, and comparing digits in images.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑