神经网络:将空间和能量守恒的概念引入神经网络
Neural Mesh: Introducing a Notion of Space and Conservation of Energy to Neural Networks
摘要:
神经网络基于大脑的简化模型。在这个项目中,我们希望通过建立一个更接近模拟神经元低层次交互的模型来放松传统神经网络的简化假设。就像RNN一样,我们的模型在时间步长之间保持一个状态,因此神经元的能量保持不变。然而,与RNN不同的是,我们的状态由2维矩阵组成,而不是1维向量,从而引入了与状态中其他神经元距离的概念。在我们的模型中,神经元只能向相邻的神经元放电,就像在大脑中一样。就像在大脑中一样,我们只允许神经元在一个时间步长内放电,如果它们包含足够的能量或兴奋。我们还强调了能量守恒的概念,因此,一个神经元不能比它在该时间步已经包含的兴奋更多地激发它的邻居。总的来说,这两个特征允许信号以激活的形式在我们的网络中随着时间的推移而流动,使我们的神经网络更紧密地模拟通过大脑的信号。虽然我们的主要目标是设计一个更接近大脑的架构,希望在我们知道如何正确训练一般智能时,拥有正确的内部信息表示,但我们确实在特定任务上对我们的神经网络进行了基准测试。我们发现,通过增加网格的运行时间,我们能够在不增加参数数量的情况下提高其精度。
英文摘要:
Neural networks are based on a simplified model of the brain. In this project, we wanted to relax the simplifying assumptions of a traditional neural network by making a model that more closely emulates the low level interactions of neurons. Like in an RNN, our model has a state that persists between time steps, so that the energies of neurons persist. However, unlike an RNN, our state consists of a 2 dimensional matrix, rather than a 1 dimensional vector, thereby introducing a concept of distance to other neurons within the state. In our model, neurons can only fire to adjacent neurons, as in the brain. Like in the brain, we only allow neurons to fire in a time step if they contain enough energy, or excitement. We also enforce a notion of conservation of energy, so that a neuron cannot excite its neighbors more than the excitement it already contained at that time step. Taken together, these two features allow signals in the form of activations to flow around in our network over time, making our neural mesh more closely model signals traveling through the brain the brain. Although our main goal is to design an architecture to more closely emulate the brain in the hope of having a correct internal representation of information by the time we know how to properly train a general intelligence, we did benchmark our neural mash on a specific task. We found that by increasing the runtime of the mesh, we were able to increase its accuracy without increasing the number of parameters.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。