arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1807.08952q-bio.NC
收藏

基于采样的概率推理是以可靠性为代价的神经电路学习

Sampling-based probabilistic inference emerges from learning in neural circuits with a cost on reliability

Laurence Aitchison, Guillaume Hennequin, Mate Lengyel

摘要:

大脑皮层中的神经反应会随着时间的推移而发生系统性的变化,这是由于持续的可塑性和学习,而由于各种噪音和变异性的来源,这种变化似乎是随机的。大多数以前的工作认为这些过程中的每一个,学习和可变性,在隔离-在这里,我们研究神经网络表现出两者,并表明它们的相互作用导致出现强大的计算性能。我们在经典的无监督学习任务上训练神经网络,其目标是以高效,易于解码的形式表示其输入,并从基本的生物物理考虑中获得神经可靠性的额外成本。这种可靠性的代价引入了能量便宜但不准确的表示与能量昂贵但准确的表示之间的权衡。尽管学习任务是非概率性的,但网络通过开发概率表示来解决这一折衷:神经变异性表示来自统计上适当的后验分布的样本,这些样本是通过对其输入进行概率推断而产生的。我们提供了一个分析的理解,这一结果揭示了可靠性的成本之间的连接,和一个国家的最先进的贝叶斯推断策略的目标:变分自动编码器。我们发现,随着网络变得越来越复杂,从单层前馈到多层前馈,再到递归架构,相同的成本会导致越来越准确的概率表示的出现。我们的研究结果为为什么感觉区域的神经反应显示出基于采样的概率表示的特征提供了见解,并可能为未来的深度学习算法及其在随机低精度计算系统中的实现提供信息。

英文摘要:

Neural responses in the cortex change over time both systematically, due to ongoing plasticity and learning, and seemingly randomly, due to various sources of noise and variability. Most previous work considered each of these processes, learning and variability, in isolation -- here we study neural networks exhibiting both and show that their interaction leads to the emergence of powerful computational properties. We trained neural networks on classical unsupervised learning tasks, in which the objective was to represent their inputs in an efficient, easily decodable form, with an additional cost for neural reliability which we derived from basic biophysical considerations. This cost on reliability introduced a tradeoff between energetically cheap but inaccurate representations and energetically costly but accurate ones. Despite the learning tasks being non-probabilistic, the networks solved this tradeoff by developing a probabilistic representation: neural variability represented samples from statistically appropriate posterior distributions that would result from performing probabilistic inference over their inputs. We provide an analytical understanding of this result by revealing a connection between the cost of reliability, and the objective for a state-of-the-art Bayesian inference strategy: variational autoencoders. We show that the same cost leads to the emergence of increasingly accurate probabilistic representations as networks become more complex, from single-layer feed-forward, through multi-layer feed-forward, to recurrent architectures. Our results provide insights into why neural responses in sensory areas show signatures of sampling-based probabilistic representations, and may inform future deep learning algorithms and their implementation in stochastic low-precision computing systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑