基于稀疏流形变换的极简无监督学习
Minimalistic Unsupervised Learning with the Sparse Manifold Transform
AI总结：
本研究提出一种基于稀疏流形变换的极简可解释无监督学习方法，无需复杂工程设计即可接近SOTA自监督方法性能，为白盒无监督学习提供了有前景的方向。
摘要：
我们提出了一种极简且可解释的无监督学习方法，无需借助数据增强、超参数调优或其他工程设计，性能即可接近当前最优（SOTA）的自监督学习（SSL）方法。该方法利用稀疏流形变换，将稀疏编码、流形学习和慢特征分析统一起来。通过单层确定性稀疏流形变换，在MNIST数据集上可实现99.3%的KNN top-1准确率，在CIFAR-10上为81.1%，在CIFAR-100上为53.2%。加入简单的灰度增强后，模型在CIFAR-10上的KNN top-1准确率达到83.2%，在CIFAR-100上达到57%。这些结果显著缩小了简单“白盒”方法与SOTA方法之间的差距。此外，我们提供了可视化结果以解释无监督表征变换的形成过程。所提方法与潜在嵌入自监督方法密切相关，可被视为VICReg的最简形式。尽管我们的简单构造模型与SOTA方法之间仍存在小幅性能差距，但相关证据表明，这是实现有理论依据的白盒无监督学习方法的一个有前景的方向。
英文摘要：
We describe a minimalistic and interpretable method for unsupervised learning, without resorting to data augmentation, hyperparameter tuning, or other engineering designs, that achieves performance close to the SOTA SSL methods. Our approach leverages the sparse manifold transform, which unifies sparse coding, manifold learning, and slow feature analysis. With a one-layer deterministic sparse manifold transform, one can achieve 99.3% KNN top-1 accuracy on MNIST, 81.1% KNN top-1 accuracy on CIFAR-10 and 53.2% on CIFAR-100. With a simple gray-scale augmentation, the model gets 83.2% KNN top-1 accuracy on CIFAR-10 and 57% on CIFAR-100. These results significantly close the gap between simplistic "white-box" methods and the SOTA methods. Additionally, we provide visualization to explain how an unsupervised representation transform is formed. The proposed method is closely connected to latent-embedding self-supervised methods and can be treated as the simplest form of VICReg. Though there remains a small performance gap between our simple constructive model and SOTA methods, the evidence points to this as a promising direction for achieving a principled and white-box approach to unsupervised learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
