随机神经网络集成中结构的涌现
Emergence of Structure in Ensembles of Random Neural Networks
AI总结:
本文提出以分类损失为能量的 Gibbs 加权随机分类器集成模型,证明存在有限最优温度,并在教师感知机设定与 MNIST 实验中展示其普适性和自组织特征。
摘要:
随机性在数据科学和机器学习的众多应用中无处不在。值得注意的是,由随机组件构成的系统往往表现出看似确定性的涌现全局行为,体现了从微观无序到宏观组织的转变。在本研究中,我们引入了一个理论模型,用于研究随机分类器集成中集体行为的涌现。我们论证指出,如果通过将分类损失作为能量而定义的 Gibbs 测度对集成进行加权,那么该分布存在一个有限温度参数,使分类在该损失(或能量)意义下达到最优。有趣的是,在样本由 Gaussian 分布生成、标签由 teacher perceptron(教师感知机)构造的情形下,我们通过解析证明并经数值验证确认,这种最优温度既不依赖于教师分类器(根据学习问题的构造,它是未知的),也不依赖于随机分类器的数量,凸显了所观察行为的普适性。在 MNIST 数据集上的实验强调了该现象在高质量、无噪声数据集中的相关性。最后,借助一个物理类比,我们阐明了所研究现象的自组织本质。
英文摘要:
Randomness is ubiquitous in many applications across data science and machine learning. Remarkably, systems composed of random components often display emergent global behaviors that appear deterministic, manifesting a transition from microscopic disorder to macroscopic organization. In this work, we introduce a theoretical model for studying the emergence of collective behaviors in ensembles of random classifiers. We argue that, if the ensemble is weighted through the Gibbs measure defined by adopting the classification loss as an energy, then there exists a finite temperature parameter for the distribution such that the classification is optimal, with respect to the loss (or the energy). Interestingly, for the case in which samples are generated by a Gaussian distribution and labels are constructed by employing a teacher perceptron, we analytically prove and numerically confirm that such optimal temperature does not depend neither on the teacher classifier (which is, by construction of the learning problem, unknown), nor on the number of random classifiers, highlighting the universal nature of the observed behavior. Experiments on the MNIST dataset underline the relevance of this phenomenon in high-quality, noiseless, datasets. Finally, a physical analogy allows us to shed light on the self-organizing nature of the studied phenomenon.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。