学习计算能力:使用神经图灵机进行二进制算术运算
Learning Numeracy: Binary Arithmetic with Neural Turing Machines
AI总结:
本文使用神经图灵机(NTMs)学习并泛化二进制加法和乘法任务,以探究该模型在更具挑战性的算法上的真实能力并缓解长期信息依赖问题。
摘要:
迄今为止,循环神经网络面临的主要问题之一是,它难以在其循环连接中保留长期的信息依赖关系。神经图灵机(NTMs)试图通过为神经网络提供一个外部记忆部分来缓解这一问题,信息可以在其中被存储并在随后进行操作。整个机制是端到端可微的,允许网络通过随机梯度下降学习如何利用这种长期记忆。这使得NTMs能够直接从数据序列中推断出简单的算法。尽管如此,由于大量的参数和相互作用的组件,该模型可能难以训练,且目前相关的研究工作很少。在这项工作中,我们使用NTMs来学习并泛化两个算术任务:二进制加法和乘法。这些任务是计算机科学中两个基本的算法示例,比以前探索过的任务要具有挑战性得多,我们希望通过这些任务来阐明这种神经模型的真实能力。
英文摘要:
One of the main problems encountered so far with recurrent neural networks is that they struggle to retain long-time information dependencies in their recurrent connections. Neural Turing Machines (NTMs) attempt to mitigate this issue by providing the neural network with an external portion of memory, in which information can be stored and manipulated later on. The whole mechanism is differentiable end-to-end, allowing the network to learn how to utilise this long-term memory via stochastic gradient descent. This allows NTMs to infer simple algorithms directly from data sequences. Nonetheless, the model can be hard to train due to a large number of parameters and interacting components and little related work is present. In this work we use NTMs to learn and generalise two arithmetical tasks: binary addition and multiplication. These tasks are two fundamental algorithmic examples in computer science, and are a lot more challenging than the previously explored ones, with which we aim to shed some light on the real capabilities on this neural model.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。