arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.15812cs.LGcs.CLcs.CV
收藏

内在维度相关性:揭示多模态表示中的非线性连接

Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations

University of Trieste(的里雅斯特大学);AREA Science Park(AREA科学园);SISSA(国际高等研究学院);MIT(麻省理工学院);ICTP(国际理论物理中心)

Lorenzo Basile, Santiago Acevedo, Luca Bortolussi, Fabio Anselmi, Alex Rodriguez

AI总结:

本文提出一种基于内在维度与相关性纠缠的度量方法,用于量化高维流形间的非线性相关性,并验证了其在多模态神经网络表示中揭示视觉与文本嵌入间非线性关联的有效性。

摘要:

为了深入理解机器学习方法背后的机制,建立描述数据点的特征之间的联系至关重要。然而,这些相关性通常表现出高维和强非线性特征,使得使用标准方法难以检测。本文利用内在维度与相关性之间的纠缠,提出了一种度量方法,用于量化高维流形之间的(潜在非线性)相关性。我们首先在受控环境中的合成数据上验证了该方法,展示了其与现有技术相比的优势和不足。随后,我们将分析扩展到神经网络表示的大规模应用中。具体而言,我们关注多模态数据的潜在表示,揭示了成对的视觉和文本嵌入之间的清晰相关性,而现有方法在检测相似性方面存在显著困难。我们的结果表明,潜在流形之间存在高度非线性的相关模式。

英文摘要:

To gain insight into the mechanisms behind machine learning methods, it is crucial to establish connections among the features describing data points. However, these correlations often exhibit a high-dimensional and strongly nonlinear nature, which makes them challenging to detect using standard methods. This paper exploits the entanglement between intrinsic dimensionality and correlation to propose a metric that quantifies the (potentially nonlinear) correlation between high-dimensional manifolds. We first validate our method on synthetic data in controlled environments, showcasing its advantages and drawbacks compared to existing techniques. Subsequently, we extend our analysis to large-scale applications in neural network representations. Specifically, we focus on latent representations of multimodal data, uncovering clear correlations between paired visual and textual embeddings, whereas existing methods struggle significantly in detecting similarity. Our results indicate the presence of highly nonlinear correlation patterns between latent manifolds.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑