发表机构
Aix-Marseille University; Basque Center on Cognition, Brain and Language; Max Planck Institute for Human Cognitive and Brain Sciences; Ikerbasque, Basque Foundation for Science（艾克斯-马赛大学; 巴斯克认知、大脑与语言中心; 马克斯·普朗克人类认知与脑科学研究所; 伊克尔巴斯基克，巴斯克科学基金会）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究通过训练RNN对带噪巴赫作品标记序列去噪，发现无额外训练约束时，RNN会自然产生预测机制，其状态含下一个标记的预测信息，可辅助感知。
AI 中文摘要
大脑非常擅长处理嘈杂且模糊的感官输入，预测加工假说认为这种能力依赖于预测，但目前尚不明确大脑为何会进化出预测感官世界的能力——这是一个计算成本高昂的过程，却能辅助感知。本文通过模拟论证，预测能力会自然出现在为感知优化的系统中：我们训练循环神经网络（RNN）对巴赫作品的标记化版本进行去噪，设置了不同的噪声水平；之后，我们探究网络的状态是否包含关于下一个标记的预测信息，具体方法是固定RNN的权重，从其状态中训练一个线性读出器用于预测，并将该线性读出器的性能与独立训练的线性基准模型的性能进行比较。结果显示，在中等噪声水平下，来自RNN的线性读出器的性能优于基准模型，表明网络依赖预测机制来支持感知；我们还进一步发现，RNN对感官输入的响应与预测误差成正比。综上，结果表明，在没有额外训练约束的情况下，预测加工的神经特征会从感知优化过程中自然出现。
英文摘要
The brain is highly proficient at making sense of noisy and ambiguous sensory inputs. Predictive processing hypothesises that this ability relies on prediction. However, it is unclear why the brain would have evolved to predict the sensory world, a computationally expensive process, in order to aid perception. Here we use simulations to argue that prediction naturally emerges in systems optimised for perception. We train recurrent neural networks (RNNs) to denoise a tokenised version of Bach's compositions at a range of noise levels. Afterwards, we enquire whether the states of the networks contain predictive information about the next token. We test this by freezing the RNN weights and training a linear readout from its states on prediction. We compare the performance of the linear readout with that of an independently trained linear benchmark model. The results show that the linear readout from the RNNs outperforms the benchmark model at moderate levels of noise, indicating that the networks rely on a predictive mechanism to support perception. We further show that the responses of the RNNs to sensory inputs are proportional to prediction error. Together, the results demonstrate that neural signatures of predictive processing emerge, without any further training constraints, from optimisation of perception.
Comments18 pages, 6 figures, 2 supplementary figures. Code and fitted weights at https://github.com/qtabs/BayesPlusBach
