首页/论文/2011.10115

arXiv 2011.101152020-11cs.LGcs.NE

Deep Neural Networks using a Single Neuron: Folded-in-Time Architecture using Feedback-Modulated Delay Loops

Institute of Mathematics, Technische Universität Berlin（柏林工业大学数学研究所）；Department of Mathematics, Humboldt-Universität zu Berlin（柏林洪堡大学数学系）；Institute of Computer Science, University of Tartu（塔尔图大学计算机科学研究所）；Instituto de Física Interdisciplinar y Sistemas Complejos, IFISC (UIB-CSIC)（跨学科物理与复杂系统研究所（巴利阿里群岛大学-西班牙高等科研理事会））

Florian Stelzer, André Röhm, Raul Vicente, Ingo Fischer, Serhiy Yanchuk

DOI：: 10.1038/s41467-021-25427-4

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By adjusting the feedback-modulation within the loops, we adapt the network's connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Reinforcement Learning with Feedback-modulated TD-STDP

University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）

Spiking neuron networks have been used successfully to solve simple reinforcement learning tasks with continuous action set applying learning rules based on spike-timing-dependent plasticity (STDP). However, most of these models cannot be applied to reinforcement learning tasks with discrete action set since they assume that the selected action is a deterministic function of firing rate of neurons, which is continuous. In this paper, we propose a new STDP-based learning rule for spiking neuron networks which contains feedback modulation. We show that the STDP-based learning rule can be used to…

2020-08-29cs.AIcs.LGstat.ML

Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity

Rutgers University（罗格斯大学）

The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, a long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the comput…

2021-10-26cs.CVcs.LGcs.NE

BioLCNet: Reward-modulated Locally Connected Spiking Neural Networks

Politecnico di Torino（都灵理工大学）；University of Tehran（德黑兰大学）；The University of Edinburgh（爱丁堡大学）；Shahid Beheshti University（沙希德·贝赫什提大学）

Brain-inspired computation and information processing alongside compatibility with neuromorphic hardware have made spiking neural networks (SNN) a promising method for solving learning tasks in machine learning (ML). Spiking neurons are only one of the requirements for building a bio-plausible learning model. Network architecture and learning rules are other important factors to consider when developing such artificial agents. In this work, inspired by the human visual pathway and the role of dopamine in learning, we propose a reward-modulated locally connected spiking neural network, BioLCNet…

2022-07-07cs.CVcs.LGcs.NE

Neural networks with motivation

Cold Spring Harbor Laboratory（冷泉港实验室）；Sainsbury Wellcome Centre（塞恩斯伯里惠康中心）；University College London（伦敦大学学院）

How can animals behave effectively in conditions involving different motivational contexts? Here, we propose how reinforcement learning neural networks can learn optimal behavior for dynamically changing motivational salience vectors. First, we show that Q-learning neural networks with motivation can navigate in environment with dynamic rewards. Second, we show that such networks can learn complex behaviors simultaneously directed towards several goals distributed in an environment. Finally, we show that in Pavlovian conditioning task, the responses of the neurons in our model resemble the fir…

2019-11-19cs.LGq-bio.NC

Neural Plasticity Networks

Georgia State University（佐治亚州立大学）

Neural plasticity is an important functionality of human brain, in which number of neurons and synapses can shrink or expand in response to stimuli throughout the span of life. We model this dynamic learning process as an $L_0$-norm regularized binary optimization problem, in which each unit of a neural network (e.g., weight, neuron or channel, etc.) is attached with a stochastic binary gate, whose parameters determine the level of activity of a unit in the network. At the beginning, only a small portion of binary gates (therefore the corresponding neurons) are activated, while the remaining n…

2021-05-02cs.LGcs.NEstat.ML

NOBLE -- 基于生物启发性潜在嵌入的神经算子：捕捉生物神经元模型中的实验变异

NOBLE -- Neural Operator with Biologically-informed Latent Embeddings to Capture Experimental Variability in Biological Neuron Models

ETH Zürich（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔人工智能研究所）；Cedars-Sinai Medical Center（Cedars-Sinai 医疗中心）；Archimedes AI, Athena Research Center（Archimedes AI，Athena 研究中心）

NOBLE通过生物启发性潜在嵌入和神经算子框架，高效生成具有实验变异性的合成神经元，实现神经动态的准确预测和模拟。

2026-02-03cs.AIcs.LGq-bio.NC

更多相关论文收起更多论文

量子光学神经元建模与基准测试以实现高效的神经计算

Modeling and benchmarking quantum optical neurons for efficient neural computation

Giuseppe Colombo Centre for Space Geodesy, Italian Space Agency（吉奥塞佩·科尔莫中心空间大地测量研究所，意大利航天局）；Department of Computer Science, University of Bari Aldo Moro（巴里阿尔多·莫罗大学计算机科学系）；Department of Physics, University of Bari Aldo Moro（巴里阿尔多·莫罗大学物理系）

本文提出基于Hong-Ou-Mandel和Mach-Zehnder干涉仪的量子光学神经元架构，通过不同光子调制策略实现高效神经计算，并在图像分类任务中验证其性能与鲁棒性。

2026-03-20cs.LGphysics.optics

Efficient single input-output layer spiking neural classifier with time-varying weight model

Nanyang Technological University（南洋理工大学）；Indian Institute of Science, Bangalore（班加罗尔印度科学学院）

This paper presents a supervised learning algorithm, namely, the Synaptic Efficacy Function with Meta-neuron based learning algorithm (SEF-M) for a spiking neural network with a time-varying weight model. For a given pattern, SEF-M uses the learning algorithm derived from meta-neuron based learning algorithm to determine the change in weights corresponding to each presynaptic spike times. The changes in weights modulate the amplitude of a Gaussian function centred at the same presynaptic spike times. The sum of amplitude modulated Gaussian functions represents the synaptic efficacy functions (…

2019-03-21cs.LGcs.NEstat.ML

利用脉冲间隔进行突触调制提高脉冲神经网络的能量效率

Synaptic Modulation using Interspike Intervals Increases Energy Efficiency of Spiking Neural Networks

Loughborough University（拉夫堡大学）；University of Leicester（莱斯特大学）

本文提出一种由脉冲间隔调制的突触模型IMSNN，利用梯度下降选择性传播梯度以增大ISI、减少脉冲发放，在保持分类准确率的同时将脉冲数量最多降低90%。

2024-08-06cs.LGcs.NE

Understanding Neural Coding on Latent Manifolds by Sharing Features and Dividing Ensembles

Norwegian University of Science and Technology（挪威科技大学）；Bosch Center for Artificial Intelligence（博世人工智能中心）；University of Cambridge（剑桥大学）

Systems neuroscience relies on two complementary views of neural data, characterized by single neuron tuning curves and analysis of population activity. These two perspectives combine elegantly in neural latent variable models that constrain the relationship between latent variables and neural activity, modeled by simple tuning curve functions. This has recently been demonstrated using Gaussian processes, with applications to realistic and topologically relevant latent manifolds. Those and previous models, however, missed crucial shared coding properties of neural populations. We propose featu…

2023-02-16cs.LGq-bio.NCstat.ML

Brain-inspired Graph Spiking Neural Networks for Commonsense Knowledge Representation and Reasoning

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）

How neural networks in the human brain represent commonsense knowledge, and complete related reasoning tasks is an important research topic in neuroscience, cognitive science, psychology, and artificial intelligence. Although the traditional artificial neural network using fixed-length vectors to represent symbols has gained good performance in some specific tasks, it is still a black box that lacks interpretability, far from how humans perceive the world. Inspired by the grandmother-cell hypothesis in neuroscience, this work investigates how population encoding and spiking timing-dependent pl…

2022-07-11cs.AIcs.LGcs.NE

Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networks

Purdue University（普渡大学）；Nanoelectronics Research Laboratory（纳米电子学研究实验室）

Stochastic gradient descent requires that training samples be drawn from a uniformly random distribution of the data. For a deployed system that must learn online from an uncontrolled and unknown environment, the ordering of input samples often fails to meet this criterion, making lifelong learning a difficult challenge. We exploit the locality of the unsupervised Spike Timing Dependent Plasticity (STDP) learning rule to target local representations in a Spiking Neural Network (SNN) to adapt to novel information while protecting essential information in the remainder of the SNN from catastroph…

2019-09-19cs.LGcs.NE

Neural Language Codes for Multilingual Acoustic Models

Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Carnegie Mellon University（卡内基梅隆大学）

Multilingual Speech Recognition is one of the most costly AI problems, because each language (7,000+) and even different accents require their own acoustic models to obtain best recognition performance. Even though they all use the same phoneme symbols, each language and accent imposes its own coloring or "twang". Many adaptive approaches have been proposed, but they require further training, additional data and generally are inferior to monolingually trained models. In this paper, we propose a different approach that uses a large multilingual model that is \emph{modulated} by the codes genera…

2018-07-05cs.CLcs.LGcs.SD