基于竞争的学习理论
Gradient-based Competitive Learning: Theory
摘要：
深度学习已被广泛用于监督学习和分类/回归问题。最近，一个新的研究领域已经将这种范式应用于无监督任务;事实上，基于梯度的方法可以有效且自主地提取用于处理输入数据的相关特征。然而，最先进的技术主要集中在算法的效率和准确性，而不是模仿输入流形。相反，竞争性学习是复制输入分布拓扑的有力工具。本文介绍了一个新的角度在这一领域结合这两种技术：无监督的基于梯度和竞争学习。该理论基于这样一种直觉，即神经网络能够通过直接对输入矩阵的转置进行操作来学习拓扑结构。为此，提出了竞争层及其对偶。前者只是对标准竞争层进行深度聚类的调整，而后者是在转置矩阵上训练的。它们的等价性在理论和实验上都得到了广泛的证明。然而，双层更适合处理非常高维的数据集。所提出的方法具有很大的潜力，因为它可以推广到大量的拓扑学习任务，如非静态和层次聚类;此外，它还可以集成到更复杂的架构中，如自动编码器和生成对抗网络。
英文摘要：
Deep learning has been widely used for supervised learning and classification/regression problems. Recently, a novel area of research has applied this paradigm to unsupervised tasks; indeed, a gradient-based approach extracts, efficiently and autonomously, the relevant features for handling input data. However, state-of-the-art techniques focus mostly on algorithmic efficiency and accuracy rather than mimic the input manifold. On the contrary, competitive learning is a powerful tool for replicating the input distribution topology. This paper introduces a novel perspective in this area by combining these two techniques: unsupervised gradient-based and competitive learning. The theory is based on the intuition that neural networks are able to learn topological structures by working directly on the transpose of the input matrix. At this purpose, the vanilla competitive layer and its dual are presented. The former is just an adaptation of a standard competitive layer for deep clustering, while the latter is trained on the transposed matrix. Their equivalence is extensively proven both theoretically and experimentally. However, the dual layer is better suited for handling very high-dimensional datasets. The proposed approach has a great potential as it can be generalized to a vast selection of topological learning tasks, such as non-stationary and hierarchical clustering; furthermore, it can also be integrated within more complex architectures such as autoencoders and generative adversarial networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
