The visual system is hierarchically organized to process visual information in successive stages. Neural representations vary drastically across the first stages of visual processing: at the output of the retina, ganglion cell receptive fields (RFs) exhibit a clear antagonistic center-surround structure, whereas in the primary visual cortex, typical RFs are sharply tuned to a precise orientation. There is currently no unified theory explaining these differences in representations across layers. Here, using a deep convolutional neural network trained on image recognition as a model of the visual system, we show that such differences in representation can emerge as a direct consequence of different neural resource constraints on the retinal and cortical networks, and we find a single model from which both geometries spontaneously emerge at the appropriate stages of visual processing. The key constraint is a reduced number of neurons at the retinal output, consistent with the anatomy of the optic nerve as a stringent bottleneck. Second, we find that, for simple cortical networks, visual representations at the retinal output emerge as nonlinear and lossy feature detectors, whereas they emerge as linear and faithful encoders of the visual scene for more complex cortices. This result predicts that the retinas of small vertebrates should perform sophisticated nonlinear computations, extracting features directly relevant to behavior, whereas retinas of large animals such as primates should mostly encode the visual scene linearly and respond to a much broader range of stimuli. These predictions could reconcile the two seemingly incompatible views of the retina as either performing feature extraction or efficient coding of natural scenes, by suggesting that all vertebrates lie on a spectrum between these two objectives, depending on the degree of neural resources allocated to their visual system.
Visual attention information can be traced on cortical response but not on the retina: evidence from electrophysiological mouse data using natural images as stimuli
Macular: a multi-scale simulation platform for the retina and the primary visual system
Université Côte d’Azur(蔚蓝海岸大学);Inria(法国国家数字科学研究所);Neuromod Institute(神经调节研究所);Service d’expérimentation et développement - SED-Inria(实验与发展服务部门-SED-Inria);Inria Rocquencourt(Inria罗坎库尔中心)
The Brain-Inspired Decoder for Natural Visual Image Reconstruction
University of Chinese Academy of Sciences(中国科学院大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions(深港脑科学创新研究院-深圳基础研究机构);Chengdu University of Traditional Chinese Medicine(成都中医药大学)