神经表征空间中的趋同进化：深度信念网络中的涌现秩序
Convergent Evolution in Neural Representation Space: Emergent Order in Deep Belief Networks
浏览论文内容
中文总结 AI 辅助
本研究发现，无监督训练的深度信念网络（DBN）可自发在表征空间中涌现类别结构，类聚类随深度增强，该效应由学习特征结构导致，而非随机变换等因素。
中文摘要 AI 辅助
深度信念网络（Deep Belief Networks, DBN）在无类别监督的情况下学习分层生成模型。本研究探讨这种纯无监督过程是否仍能根据未知数据类别组织内部表征。我们使用广义判别值（Generalized Discrimination Value, GDV）、仅在训练后应用的监督探针、基于重构的抽象距离度量、有效维度以及自由样本生成，分析在MNIST、Fashion-MNIST和KMNIST数据集上训练的DBN的连续层。值得注意的是，在DBN训练过程中无任何标签信息的情况下，跨数据集和网络宽度，特定类别的聚类通常随深度增加而增强。对照实验表明，该效应依赖于学习到的特征结构，无法用随机变换、权重边际、降维或sigmoid饱和来解释。第一隐藏层还常使线性和非线性探针更易获取类别身份。随着深度增加，表征变得愈发紧凑且类原型化，因为神经元获得了相关特征方向。同时，GDV和探针准确率揭示了类别结构的互补方面：平均聚类的改善可与少数困难类别对的可访问性降低并存。这些发现表明，分层生成学习能自发揭示并逐步放大无标签数据中与类别相关的结构。
英文摘要
Deep Belief Networks (DBNs) learn hierarchical generative models without class supervision. Here, we ask whether this purely unsupervised process nevertheless organizes internal representations according to the unknown data classes. We analyze successive layers of DBNs trained on MNIST, Fashion-MNIST, and KMNIST using the Generalized Discrimination Value (GDV), supervised probes applied only after training, a reconstruction-based measure of abstraction distance, effective dimensionality, and free sample generation. Remarkably, class-specific clustering generally increases with depth across datasets and network widths, although no label information is available during DBN training. Control experiments show that this effect depends on the learned feature structure and cannot be explained by random transformations, weight marginals, dimensionality reduction, or sigmoid saturation. The first hidden layers also frequently make class identity more accessible to linear and nonlinear probes. With greater depth, representations become increasingly compact and prototype-like as neurons acquire correlated feature directions. At the same time, GDV and probe accuracy reveal complementary aspects of class structure: improved average clustering can coexist with reduced accessibility for a few difficult class pairs. These findings demonstrate that layer-wise generative learning can spontaneously uncover and progressively amplify class-related structure in unlabeled data.
