arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.09485cs.LGstat.ML
收藏

SEMASIA:一个大规模的语义结构化潜在表示数据集

SEMASIA: A Large-Scale Dataset of Semantically Structured Latent Representations

Dept. Computer, Control and Management Engineering(计算机、控制与管理工程系);Sapienza University of Rome(罗马大学西皮恩扎分校);National Inter-University Consortium for Telecommunications (CNIT)(电信全国大学联合体(CNIT));Dept. of Statistical Sciences(统计科学系);Dept. of Information Engineering, Electronics, and Telecommunications(信息工程、电子与电信系)

Mario Edoardo Pandolfo, Enrico Grimaldi, Lorenzo Marinucci, Leonardo Di Nino, Simone Fiorellino, Sergio Barbarossa, Paolo Di Lorenzo

AI总结:

SEMASIA通过收集1700多个预训练视觉模型的潜在表示,提供语义结构化的元数据,用于分析潜在空间组织、对齐映射基准测试及回归分析预训练数据复杂性与嵌入几何特性之间的关系。

摘要:

由神经网络学习的潜在表示通常表现出语义结构,其中概念相似性通过嵌入空间中的几何接近性反映。然而,跨模型比较这些空间仍然困难:架构变化、预训练数据、目标或随机种子的变化可能导致内容相似但几何不兼容的嵌入。这个潜在空间对齐问题对可解释性、迁移和多模态学习、联邦系统和语义通信至关重要,但进展受限于缺乏大规模、模型多样且元数据丰富的基准。为解决这一差距,我们引入了SEMASIA,一个大规模的潜在表示集合,从大约1700个预训练视觉模型中提取,覆盖八个标准图像分类基准。SEMASIA将嵌入与描述架构、训练制度、预训练来源和模型规模的结构化元数据配对。我们展示了该资源的三个应用:首先,分析单个潜在空间的概念组织,显示在模型和数据集之间的一致原型样聚类和层次语义邻域;其次,使用重建误差和下游任务性能基准测试潜在空间之间的监督对齐映射;第三,进行大规模回归分析,探讨预训练数据复杂性、专业化、迁移学习、增强和模型规模如何与嵌入的几何和探测特性相关联。通过将表示规模与标准化元数据结合,SEMASIA为研究潜在几何、评估对齐方法以及开发下一代异构和互操作的AI系统提供了可重复的基础。

英文摘要:

Latent representations learned by neural networks often exhibit semantic structure, where concept similarity is reflected by geometric proximity in embedding space. However, comparing such spaces across models remains difficult: changes in architecture, pretraining data, objective, or random seed can yield embeddings with similar content but incompatible geometry. This latent space alignment problem is central to interpretability, transfer and multimodal learning, federated systems, and semantic communication; however, progress remains limited by the lack of large-scale, model-diverse, and metadata-rich benchmarks. To address this gap, we introduce SEMASIA, a large-scale collection of latent representations extracted from approximately 1,700 pretrained vision models across eight standard image-classification benchmarks. SEMASIA pairs embeddings with structured metadata describing architectures, training regimes, pretraining sources, and model scale. We demonstrate three applications of the resource. First, we analyze the conceptual organization of individual latent spaces, showing consistent prototype-like clustering and hierarchical semantic neighborhoods across models and datasets. Second, we benchmark supervised alignment mappings between latent spaces using reconstruction error and downstream task performance. Third, we perform a large-scale regression analysis of how pretraining-data complexity, specialization, transfer learning, augmentation, and model scale relate to geometric and probing properties of embeddings. By coupling representational scale with standardized metadata, SEMASIA provides a reproducible foundation for studying latent geometry, evaluating alignment methods, and developing next-generation heterogeneous and interoperable AI systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑