arXiv 1710.095742017-10-26stat.MLcs.LGcs.NE

Biologically Inspired Feedforward Supervised Learning for Deep Self-Organizing Map Networks

CiNet, National Institute of Information and Communications Technology（信息通信研究机构脑信息通信融合研究中心）；Graduate School of Information Science and Technology, Osaka University（大阪大学信息科学与技术研究科）

Takashi Shinozaki

备注：: Presented at MLINI-2016 workshop, 2016 (arXiv:1701.01437)

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

In this study, we propose a novel deep neural network and its supervised learning method that uses a feedforward supervisory signal. The method is inspired by the human visual system and performs human-like association-based learning without any backward error propagation. The feedforward supervisory signal that produces the correct result is preceded by the target signal and associates its confirmed label with the classification result of the target signal. It effectively uses a large amount of information from the feedforward signal, and forms a continuous and rich learning representation. The method is validated using visual recognition tasks on the MNIST handwritten dataset.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Self-Organizing Maps with Variable Input Length for Motif Discovery and Word Segmentation

Universidade Federal de Pernambuco（伯南布哥联邦大学）

Time Series Motif Discovery (TSMD) is defined as searching for patterns that are previously unknown and appear with a given frequency in time series. Another problem strongly related with TSMD is Word Segmentation. This problem has received much attention from the community that studies early language acquisition in babies and toddlers. The development of biologically plausible models for word segmentation could greatly advance this field. Therefore, in this article, we propose the Variable Input Length Map (VILMAP) for Motif Discovery and Word Segmentation. The model is based on the Self-Orga…

2019-08-07cs.LGcs.NEstat.ML

Variational Probability Flow for Biologically Plausible Training of Deep Neural Networks

Singapore University of Technology and Design（新加坡科技设计大学）

The quest for biologically plausible deep learning is driven, not just by the desire to explain experimentally-observed properties of biological neural networks, but also by the hope of discovering more efficient methods for training artificial networks. In this paper, we propose a new algorithm named Variational Probably Flow (VPF), an extension of minimum probability flow for training binary Deep Boltzmann Machines (DBMs). We show that weight updates in VPF are local, depending only on the states and firing rates of the adjacent neurons. Unlike contrastive divergence, there is no need for Gi…

2017-11-21cs.LGcs.NEstat.ML

A Neural Spiking Approach Compared to Deep Feedforward Networks on Stepwise Pixel Erasement

Chemnitz University of Technology（开姆尼茨工业大学）

In real world scenarios, objects are often partially occluded. This requires a robustness for object recognition against these perturbations. Convolutional networks have shown good performances in classification tasks. The learned convolutional filters seem similar to receptive fields of simple cells found in the primary visual cortex. Alternatively, spiking neural networks are more biological plausible. We developed a two layer spiking network, trained on natural scenes with a biologically plausible learning rule. It is compared to two deep convolutional neural networks using a classification…

2019-12-06cs.CVcs.LGcs.NE

Backprop Diffusion is Biologically Plausible

University of Siena（锡耶纳大学）

The Backpropagation algorithm relies on the abstraction of using a neural model that gets rid of the notion of time, since the input is mapped instantaneously to the output. In this paper, we claim that this abstraction of ignoring time, along with the abrupt input changes that occur when feeding the training set, are in fact the reasons why, in some papers, Backprop biological plausibility is regarded as an arguable issue. We show that as soon as a deep feedforward network operates with neurons with time-delayed response, the backprop weight update turns out to be the basic equation of a biol…

2020-06-17cs.LGcs.NEq-bio.NC

Ghost Units Yield Biologically Plausible Backprop in Deep Neural Networks

Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Universite de Montreal（蒙特利尔大学）；Institute of Neuroinformatics（神经信息学研究所）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；University of Bern（伯尔尼大学）

In the past few years, deep learning has transformed artificial intelligence research and led to impressive performance in various difficult tasks. However, it is still unclear how the brain can perform credit assignment across many areas as efficiently as backpropagation does in deep neural networks. In this paper, we introduce a model that relies on a new role for a neuronal inhibitory machinery, referred to as ghost units. By cancelling the feedback coming from the upper layer when no target signal is provided to the top layer, the ghost units enables the network to backpropagate errors and…

2019-11-15cs.LGq-bio.NCstat.ML

Contrastive Similarity Matching for Supervised Learning

Harvard University（哈佛大学）

We propose a novel biologically-plausible solution to the credit assignment problem motivated by observations in the ventral visual pathway and trained deep neural networks. In both, representations of objects in the same category become progressively more similar, while objects belonging to different categories become less similar. We use this observation to motivate a layer-specific learning goal in a deep network: each layer aims to learn a representational similarity matrix that interpolates between previous and later layers. We formulate this idea using a contrastive similarity matching o…

2020-12-06cs.LGq-bio.NCstat.ML

更多相关论文收起更多论文

Deep learning with asymmetric connections and Hebbian updates

University of Chicago（芝加哥大学）

We show that deep networks can be trained using Hebbian updates yielding similar performance to ordinary back-propagation on challenging image datasets. To overcome the unrealistic symmetry in connections between layers, implicit in back-propagation, the feedback weights are separate from the feedforward weights. The feedback weights are also updated with a local rule, the same as the feedforward weights - a weight is updated solely based on the product of activity of the units it connects. With fixed feedback weights as proposed in Lillicrap et. al (2016) performance degrades quickly as the d…

2019-03-12cs.LGcs.NEq-bio.NC

Oja可塑性规则克服了在生物约束下训练神经网络的若干挑战

Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints

本研究提出将Oja可塑性规则融入误差驱动训练，在前馈和循环架构中实现稳定高效学习，无需复杂工程技巧，并在数据稀缺场景下超越标准反向传播，为连接深度网络与生物突触机制提供了路径。

2025-04-05q-bio.NCstat.ML

Feedback alignment in deep convolutional networks

Columbia University（哥伦比亚大学）

Ongoing studies have identified similarities between neural representations in biological networks and in deep artificial neural networks. This has led to renewed interest in developing analogies between the backpropagation learning algorithm used to train artificial networks and the synaptic plasticity rules operative in the brain. These efforts are challenged by biologically implausible features of backpropagation, one of which is a reliance on symmetric forward and backward synaptic weights. A number of methods have been proposed that do not rely on weight symmetry but, thus far, these have…

2019-06-10cs.LGcs.NEstat.ML