物理前馈神经网络中的噪声抑制策略
Noise mitigation strategies in physical feedforward neural networks
摘要：
物理神经网络是下一代人工智能硬件的有前途的候选者。在这样的架构中，神经元和连接是物理实现的，并且不利用具有实际上无限信噪比的数字概念来编码、解码和转换信息。因此，它们容易受到具有各种统计和架构属性的噪声的影响，并且在追求下一代神经网络硬件的过程中，利用网络固有资产以硬件高效的方式减轻噪声的有效策略是重要的。基于分析推导，我们在这里介绍和分析各种不同的降噪方法。我们分析表明，层内连接，其中连接矩阵的平方均数超过其平方的平均值完全抑制不相关的噪声。我们超越并开发了两种协同策略，用于在神经元群体中不相关和相关的噪声。首先，我们介绍了幽灵神经元的概念，其中被相关噪声干扰的每组神经元与单个神经元有负连接，但没有接收任何输入信息。其次，我们表明，池的神经元群体是一种有效的方法来抑制不相关的噪声。因此，我们开发了一种通用的噪声缓解策略，利用模拟硬件中最相关的不同噪声项的统计特性。最后，我们证明了这种组合方法的有效性训练神经网络分类MNIST手写数字，我们实现了4倍的输出信噪比的改善，并增加了分类精度几乎无噪声网络的水平。
英文摘要：
Physical neural networks are promising candidates for next generation artificial intelligence hardware. In such architectures, neurons and connections are physically realized and do not leverage digital concepts with their practically infinite signal-to-noise ratio to encode, transduce and transform information. They therefore are prone to noise with a variety of statistical and architectural properties, and effective strategies leveraging network-inherent assets to mitigate noise in an hardware-efficient manner are important in the pursuit of next generation neural network hardware. Based on analytical derivations, we here introduce and analyse a variety of different noise-mitigation approaches. We analytically show that intra-layer connections in which the connection matrix's squared mean exceeds the mean of its square fully suppresses uncorrelated noise. We go beyond and develop two synergistic strategies for noise that is uncorrelated and correlated across populations of neurons. First, we introduce the concept of ghost neurons, where each group of neurons perturbed by correlated noise has a negative connection to a single neuron, yet without receiving any input information. Secondly, we show that pooling of neuron populations is an efficient approach to suppress uncorrelated noise. As such, we developed a general noise mitigation strategy leveraging the statistical properties of the different noise terms most relevant in analogue hardware. Finally, we demonstrate the effectiveness of this combined approach for trained neural network classifying the MNIST handwritten digits, for which we achieve a 4-fold improvement of the output signal-to-noise ratio and increase the classification accuracy almost to the level of the noise-free network.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
