Shannon Information Capacity of Discrete Synapses

Synaptic Transmission: An Information-Theoretic Perspective

Here we analyze synaptic transmission from an information-theoretic perspective. We derive closed-form expressions for the lower-bounds on the capacity of a simple model of a cortical synapse under two explicit coding paradigms. Under the ``signal estimation'' paradigm, we assume the signal to be encoded in the mean firing rate of a Poisson neuron. The performance of an optimal linear estimator of the signal then provides a lower bound on the capacity for signal estimation. Under the ``signal detection'' paradigm, the presence or absence of the signal has to be detected. Performance of the opt…

Synaptic efficacy shapes resource limitations in working memory

Working memory (WM) is limited in its temporal length and capacity. Classic conceptions of WM capacity assume the system possesses a finite number of slots, but recent evidence suggests WM may be a continuous resource. Resource models typically assume there is no hard upper bound on the number of items that can be stored, but WM fidelity decreases with the number of items. We analyze a neural field model of multi-item WM that associates each item with the location of a bump in a finite spatial domain, considering items that span a one-dimensional continuous feature space. Our analysis relates …

Optimal Learning with Excitatory and Inhibitory synapses

Characterizing the relation between weight structure and input/output statistics is fundamental for understanding the computational capabilities of neural circuits. In this work, I study the problem of storing associations between analog signals in the presence of correlations, using methods from statistical mechanics. I characterize the typical learning performance in terms of the power spectrum of random input and output processes. I show that optimal synaptic weight configurations reach a capacity of 0.5 for any fraction of excitatory to inhibitory weights and have a peculiar synaptic distr…

分数阶离散时间系统的稳定性及其在 Rulkov 神经网络和非对称忆阻器突触中的应用

Stability of Fractional-Order Discrete-Time Systems with Application to Rulkov Neural Networks and Asymmetric Memristor Synapses

针对忆阻器耦合分数阶离散神经模型研究不足，提出两类 Rulkov 神经元系统并给出通用稳定性区域定理。

A three-threshold learning rule approaches the maximal capacity of recurrent neural networks

Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model has a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error s…

An attractor neural network architecture with an ultra high information capacity: numerical results

Attractor neural network is an important theoretical scenario for modeling memory function in the hippocampus and in the cortex. In these models, memories are stored in the plastic recurrent connections of neural populations in the form of "attractor states". The maximal information capacity for conventional abstract attractor networks with unconstrained connections is 2 bits/synapse. However, an unconstrained synapse has the capacity to store infinite amount of bits in a noiseless theoretical scenario: a capacity that conventional attractor networks cannot achieve. Here, I propose a hierarchi…

量化神经网络突触连接中存储的信息而非神经元放电活动

Quantifying information stored in synaptic connections rather than in firing activities of neural networks

本文提出了一种理论框架，用于量化神经网络突触连接分布中存储的信息，通过Hebbian网络和对数正态分布建模，揭示突触间协同作用及信息存储的双重性。