arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.04549cs.NEcs.LGmath.FAstat.ML
收藏

使用配备仿射编码器和解码器的脉冲神经网络进行稳定学习

Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoders

University of Vienna(维也纳大学);Faculty of Mathematics(数学系)

A. Martina Neuman, Dominik Dold, Philipp Christian Petersen

AI总结:

本研究提出仿射脉冲神经网络,通过正权重与仿射编解码器实现稳定学习,能逼近ReLU网络,且在基准测试中表现优异,深度不影响泛化。

摘要:

我们研究了与脉冲神经网络相关的学习问题。具体而言,我们关注由仅具有正突触权重的简单脉冲神经元组成的脉冲神经网络,并配备仿射编码器和解码器;我们将其称为仿射脉冲神经网络。我们证明了这些神经网络对其参数的连续依赖性,这有助于基于经典覆盖数的泛化表述,并支持稳定的基于梯度的训练。我们证明了权重的正性使得广泛的表达能力结果成为可能,包括光滑函数的最优速率逼近以及Barron正则函数的维度无关逼近。特别地,我们在理论和模拟中表明,仿射脉冲神经网络能够逼近浅层ReLU神经网络。此外,我们将这些仿射脉冲神经网络应用于标准机器学习基准测试,并取得了具有竞争力的结果。最后,我们从泛化的角度观察到,与前馈神经网络或先前针对一般脉冲神经网络的结果相反,深度对泛化能力几乎没有不利影响。

英文摘要:

We study the learning problem associated with spiking neural networks. Specifically, we focus on spiking neural networks composed of simple spiking neurons having only positive synaptic weights, equipped with an affine encoder and decoder; we refer to these as affine spiking neural networks. These neural networks are shown to depend continuously on their parameters, which facilitates classical covering number-based generalization statements and supports stable gradient-based training. We demonstrate that the positivity of the weights enables a wide range of expressivity results, including rate-optimal approximation of smooth functions and dimension-independent approximation of Barron regular functions. In particular, we show in theory and simulations that affine spiking neural networks are capable of approximating shallow ReLU neural networks. Furthermore, we apply these affine spiking neural networks to standard machine learning benchmarks and reach competitive results. Finally, we observe that from a generalization perspective, contrary to feedforward neural networks or previous results for general spiking neural networks, the depth has little to no adverse effect on the generalization capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑