首页/论文/2001.01121

arXiv 2001.011212020-01-04cs.NEcs.CVcs.LG

Biologically-Motivated Deep Learning Method using Hierarchical Competitive Learning

Center for Information and Neural Networks (CiNet)（信息与神经网络中心）；National Institute of Information and Communications Technology (NICT)（日本信息通信技术研究所）；Graduate School of Information Science and Technology, Osaka University（大阪大学信息科学与技术研究生院）

Takashi Shinozaki

备注：: Appeared at NIPS 2019 Workshop: Shared Visual Representations in Human and Machine Intelligence (SVRHM)

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

This study proposes a novel biologically-motivated learning method for deep convolutional neural networks (CNNs). The combination of CNNs and back propagation (BP) learning is the most powerful method in recent machine learning regimes. However, it requires large labeled data for training, and this requirement can occasionally become a barrier for real world applications. To address this problem and utilize unlabeled data, I propose to introduce unsupervised competitive learning which only requires forward propagating signals as a pre-training method for CNNs. The method was evaluated by image discrimination tasks using MNIST, CIFAR-10, and ImageNet datasets, and it achieved a state-of-the-art performance as a biologically-motivated method in the ImageNet experiment. The results suggested that the method enables higher-level learning representations solely from forward propagating signals without a backward error signal for the learning of convolutional layers. The proposed method could be useful for a variety of poorly labeled data, for example, time series or medical data.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

层级时间感受窗口与生物受限尺度不变深度网络中的零样本时间尺度泛化

Hierarchical temporal receptive windows and zero-shot timescale generalization in biologically constrained scale-invariant deep networks

Department of Psychological and Brain Sciences（心理与脑科学系）；Department of Physics（物理系）

该研究通过生物受限的深度网络，探索了尺度不变的递归架构在语言分类任务中的零样本时间尺度泛化能力，揭示了人类认知中层级时间感受窗口的作用。

2026-01-06cs.AIcs.CLcs.LG

Are skip connections necessary for biologically plausible learning rules?

Janelia Research Campus（珍妮亚研究园区）；HHMI（霍华德·休斯医学研究所）

Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these methods have produced a competitive performance against backpropagation. In this paper, we show that biologically-motivated learning rules with skip connections between intermediate layers can perform as well as backpropagation on the MNIST dataset and are robust to various sets of hyper-parameters.

2019-12-04cs.LGcs.NEstat.ML

面向计算机视觉应用的生物启发式深度残差网络

Biologically inspired deep residual networks for computer vision applications

Noorul Islam Centre for Higher Education（努尔伊斯兰高等教育中心）

本文提出在ResNet跳跃连接中引入六边形卷积的生物启发式深度残差网络，在ImageNet 2012和CIFAR-10上分别将基线top-1准确率平均提升1.35%和0.48%，显示出更好的泛化性能。

2022-05-05cs.CVcs.NEeess.IV

Physical Deep Learning with Biologically Plausible Training Method

NTT Device Technology Labs.（NTT设备技术实验室）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息理工学系研究科）；Next Generation Artificial Intelligence Research Center, The University of Tokyo（东京大学下一代人工智能研究中心）

The ever-growing demand for further advances in artificial intelligence motivated research on unconventional computation based on analog physical devices. While such computation devices mimic brain-inspired analog information processing, learning procedures still relies on methods optimized for digital processing such as backpropagation. Here, we present physical deep learning by extending a biologically plausible training algorithm called direct feedback alignment. As the proposed method is based on random projection with arbitrary nonlinear activation, we can train a physical neural network …

2022-04-01cs.LGcs.NEnlin.AO

Classifier with Hierarchical Topographical Maps as Internal Representation

In this study we want to connect our previously proposed context-relevant topographical maps with the deep learning community. Our architecture is a classifier with hidden layers that are hierarchical two-dimensional topographical maps. These maps differ from the conventional self-organizing maps in that their organizations are influenced by the context of the data labels in a top-down manner. In this way bottom-up and top-down learning are combined in a biologically relevant representational learning setting. Compared to our previous work, we are here specifically elaborating the model in a m…

2015-04-03cs.NE

Towards efficient end-to-end speech recognition with biologically-inspired neural networks

Axelera AI；IBM Research, Zurich（IBM苏黎世研究院）；Graz University of Technology（格拉茨工业大学）；ETH Zurich（苏黎世联邦理工学院）；IBM Research AI, Yorktown Heights（IBM研究院）

Automatic speech recognition (ASR) is a capability which enables a program to process human speech into a written form. Recent developments in artificial intelligence (AI) have led to high-accuracy ASR systems based on deep neural networks, such as the recurrent neural network transducer (RNN-T). However, the core components and the performed operations of these approaches depart from the powerful biological counterpart, i.e., the human brain. On the other hand, the current developments in biologically-inspired ASR models, based on spiking neural networks (SNNs), lag behind in terms of accurac…

2021-11-04cs.LGcs.NEeess.AS

更多相关论文收起更多论文

Low-Order Model of Biological Neural Networks

A biologically plausible low-order model (LOM) of biological neural networks is a recurrent hierarchical network of dendritic nodes/trees, spiking/nonspiking neurons, unsupervised/ supervised covariance/accumulative learning mechanisms, feedback connections, and a scheme for maximal generalization. These component models are motivated and necessitated by making LOM learn and retrieve easily without differentiation, optimization, or iteration, and cluster, detect and recognize multiple/hierarchical corrupted, distorted, and occluded temporal and spatial patterns.

2020-12-12cs.NEmath.OCq-bio.NC

Minimizing Control for Credit Assignment with Strong Feedback

Institute of Neuroinformatics（神经信息学研究所）；University of Zürich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）

The success of deep learning ignited interest in whether the brain learns hierarchical representations using gradient-based learning. However, current biologically plausible methods for gradient-based credit assignment in deep neural networks need infinitesimally small feedback signals, which is problematic in biologically realistic noisy environments and at odds with experimental evidence in neuroscience showing that top-down feedback can significantly influence neural activity. Building upon deep feedback control (DFC), a recently proposed credit assignment method, we combine strong feedback…

2022-06-22cs.LGcs.NE

Unsupervised Regenerative Learning of Hierarchical Features in Spiking Deep Networks for Object Recognition

We present a spike-based unsupervised regenerative learning scheme to train Spiking Deep Networks (SpikeCNN) for object recognition problems using biologically realistic leaky integrate-and-fire neurons. The training methodology is based on the Auto-Encoder learning model wherein the hierarchical network is trained layer wise using the encoder-decoder principle. Regenerative learning uses spike-timing information and inherent latencies to update the weights and learn representative levels for each convolutional layer in an unsupervised manner. The features learnt from the final layer in the hi…

2016-02-03cs.NE

FOVI：一种受生物启发的深度视觉模型中央凹接口

FOVI: A biologically-inspired foveated interface for deep vision models

harvard（哈佛大学）；nvidia

受人类视觉系统启发，提出基于视网膜和V1的中央凹接口FOVI，通过kNN卷积和低秩适应实现高效变分辨率视觉处理，在减少像素和计算成本的同时保持竞争力。

2026-05-29cs.CVcs.NEq-bio.NC

Encoding Data for HTM Systems

Hierarchical Temporal Memory (HTM) is a biologically inspired machine intelligence technology that mimics the architecture and processes of the neocortex. In this white paper we describe how to encode data as Sparse Distributed Representations (SDRs) for use in HTM systems. We explain several existing encoders, which are available through the open source project called NuPIC, and we discuss requirements for creating encoders for new types of data.

2016-02-18cs.NEq-bio.NC

Convolutional Spike Timing Dependent Plasticity based Feature Learning in Spiking Neural Networks

Purdue University（普渡大学）

Brain-inspired learning models attempt to mimic the cortical architecture and computations performed in the neurons and synapses constituting the human brain to achieve its efficiency in cognitive tasks. In this work, we present convolutional spike timing dependent plasticity based feature learning with biologically plausible leaky-integrate-and-fire neurons in Spiking Neural Networks (SNNs). We use shared weight kernels that are trained to encode representative features underlying the input patterns thereby improving the sparsity as well as the robustness of the learning model. We demonstrate…

2017-03-20cs.AIcs.CVcs.NE

自监督局部学习规则学习高维数据的隐藏层次结构

Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data

School of Life Science and School of Computer and Communications Sciences, EPFL（生命科学学院和计算机与通信科学学院，瑞士联邦理工学院）；Machine Learning Research Unit, TU Wien（机器学习研究单位，维也纳技术大学）

本文研究了自监督局部学习规则在随机层次模型上的表现，发现第一类规则因输入特定的非线性（masking）失效，而第二类规则能有效学习层次结构并具备数据效率和生物合理性。

2026-05-18cs.LGcs.NEq-bio.NC

Deep Reservoir Networks with Learned Hidden Reservoir Weights using Direct Feedback Alignment

University of Maryland（马里兰大学）；Institute for Adv. Computer Studies（高级计算机研究所）

Deep Reservoir Computing has emerged as a new paradigm for deep learning, which is based around the reservoir computing principle of maintaining random pools of neurons combined with hierarchical deep learning. The reservoir paradigm reflects and respects the high degree of recurrence in biological brains, and the role that neuronal dynamics play in learning. However, one issue hampering deep reservoir network development is that one cannot backpropagate through the reservoir layers. Recent deep reservoir architectures do not learn hidden or hierarchical representations in the same manner as d…

2020-10-15cs.LGcs.NE

Sparsey: Event Recognition via Deep Hierarchical Spare Distributed Codes

Neurithmic Systems LLC（Neurithmic Systems 有限责任公司）

Visual cortex's hierarchical, multi-level organization is captured in many biologically inspired computational vision models, the general idea being that progressively larger scale, more complex spatiotemporal features are represented in progressively higher areas. However, most earlier models use localist representations (codes) in each representational field, which we equate with the cortical macrocolumn (mac), at each level. In localism, each represented feature/event (item) is coded by a single unit. Our model, Sparsey, is also hierarchical but crucially, uses sparse distributed coding (SD…

2016-11-12cs.CVcs.NEq-bio.NC