Categorical Perception：深度学习的基础
Categorical Perception: A Groundwork for Deep Learning
摘要：
在人类和其他动物中，分类的一个众所周知的感知结果，称为类别感知，其显著特征是类别内压缩和类别间分离：两个项目，在输入空间中接近，如果它们属于同一类别，则被感知为更接近。在阐述认知科学的实验和理论结果的基础上，我们在这里研究人工神经网络中的分类效应。我们结合了理论分析，利用相互和Fisher信息量，和一系列的数值模拟网络的日益复杂。这些形式和数值分析提供了深入了解深层神经表征的几何结构，在类别边界附近的空间扩展和远离类别边界的收缩。我们调查的范畴表示使用两种互补的方法：一个模仿心理物理学和认知神经科学的实验，通过不同类别的刺激之间的变形连续，而另一个引入了一个分类指数，对于网络中的每一层，量化的类别在神经种群水平的可分性。我们在浅层和深层神经网络上证明了类别学习会自动诱导类别感知。我们进一步表明，层越深，分类效应越强。作为我们研究的结果，我们提出了一个连贯的观点，不同的启发式实践的辍学正则化技术的功效。更一般地说，我们的观点在神经科学文献中得到了回应，坚持认为噪声在任何给定层中的差异影响取决于正在学习的神经表征的几何结构，即这种几何结构如何反映类别的结构。
英文摘要：
A well-known perceptual consequence of categorization in humans and other animals, called categorical perception, is notably characterized by a within-category compression and a between-category separation: two items, close in input space, are perceived closer if they belong to the same category than if they belong to different categories. Elaborating on experimental and theoretical results in cognitive science, here we study categorical effects in artificial neural networks. We combine a theoretical analysis that makes use of mutual and Fisher information quantities, and a series of numerical simulations on networks of increasing complexity. These formal and numerical analyses provide insights into the geometry of the neural representation in deep layers, with expansion of space near category boundaries and contraction far from category boundaries. We investigate categorical representation by using two complementary approaches: one mimics experiments in psychophysics and cognitive neuroscience by means of morphed continua between stimuli of different categories, while the other introduces a categoricality index that, for each layer in the network, quantifies the separability of the categories at the neural population level. We show on both shallow and deep neural networks that category learning automatically induces categorical perception. We further show that the deeper a layer, the stronger the categorical effects. As an outcome of our study, we propose a coherent view of the efficacy of different heuristic practices of the dropout regularization technique. More generally, our view, which finds echoes in the neuroscience literature, insists on the differential impact of noise in any given layer depending on the geometry of the neural representation that is being learned, i.e. on how this geometry reflects the structure of the categories.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
