URLOST：无平稳性或拓扑结构的无监督表征学习
URLOST: Unsupervised Representation Learning without Stationarity or Topology
AI总结：
针对现有无监督表征学习依赖领域平稳性与拓扑结构的局限，提出结合可学习自组织层、谱聚类和MAE的URLOST框架，在多模态数据上性能超越现有方法，推动跨模态泛化的无监督学习发展。
摘要：
无监督表征学习已取得巨大进展，但它受限于对特定领域平稳性和拓扑结构的依赖，这一局限在生物智能系统中并不存在。例如，与计算机视觉不同，人类视觉能够处理从高度不规则、非平稳传感器采样得到的视觉信号。\n我们提出了一个无需平稳性和拓扑结构先验知识即可从高维数据中学习的新型框架，模型缩写为URLOST，结合了可学习自组织层、谱聚类和掩码自编码器（masked autoencoder, MAE）。\n我们在三种不同数据模态上评估了其有效性，包括模拟生物视觉数据、初级视皮层神经记录以及基因表达数据。与SimCLR、MAE等当前最优的无监督学习方法相比，我们的模型在无需知晓数据平稳性或拓扑结构的情况下，擅长跨多种模态学习有意义的表征，其表现也优于其他不依赖这些因素的方法，为该领域设定了新基准。我们将这项工作定位为迈向可跨多种高维数据模态泛化的无监督学习方法的一步。
英文摘要：
Unsupervised representation learning has seen tremendous progress. However, it is constrained by its reliance on domain specific stationarity and topology, a limitation not found in biological intelligence systems. For instance, unlike computer vision, human vision can process visual signals sampled from highly irregular and non-stationary sensors. We introduce a novel framework that learns from high-dimensional data without prior knowledge of stationarity and topology. Our model, abbreviated as URLOST, combines a learnable self-organizing layer, spectral clustering, and a masked autoencoder (MAE). We evaluate its effectiveness on three diverse data modalities including simulated biological vision data, neural recordings from the primary visual cortex, and gene expressions. Compared to state-of-the-art unsupervised learning methods like SimCLR and MAE, our model excels at learning meaningful representations across diverse modalities without knowing their stationarity or topology. It also outperforms other methods that are not dependent on these factors, setting a new benchmark in the field. We position this work as a step toward unsupervised learning methods capable of generalizing across diverse high-dimensional data modalities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
