arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.04458stat.MLphysics.data-an
收藏

同步降维:一种用于多模态表示学习的数据高效方法

Simultaneous Dimensionality Reduction: A Data Efficient Approach for Multimodal Representations Learning

Eslam Abdelaleem, Ahmed Roman, K. Michael Martini, Ilya Nemenman

AI总结:

本文提出生成式线性模型与数值实验比较IDR和SDR,证明线性SDR尤其正则化CCA能在小样本、高维场景下更高效地检测多模态协变结构。

摘要:

我们探讨了降维(DR)的两大类主要方法:独立降维(IDR)和同步降维(SDR)。在IDR方法中,主成分分析是一个典型示例,每种模态都被独立压缩,力求保留每种模态内尽可能多的变异。相比之下,在SDR中,人们同时压缩各模态,以最大化降维描述之间的协变,而较少关注保留了多少个体变异。典型示例包括偏最小二乘法和典型相关分析。尽管这些DR方法是统计学中的常用工具,但人们对它们的相对准确性和数据集规模要求仍知之甚少。我们引入了一个生成式线性模型,用于合成具有已知方差和协方差结构的多模态数据,以考察这些问题。我们将协方差结构重建的准确性评估为样本数量、信噪比以及数据中变化信号和协变信号数量的函数。通过数值实验,我们证明线性SDR方法始终优于线性IDR方法,并能在更小的数据集上产生质量更高、更简洁的降维表示。值得注意的是,即使样本数量远小于数据维度——这是一个对所有降维方法都具有挑战性的区间——正则化CCA也能够识别低维弱协变结构。我们的工作证实并解释了文献中先前的观察,即SDR在检测数据中的协变模式方面可能更加有效。这些发现表明,在现实世界数据分析中,当检测协变比保留变异更重要时,应优先选择SDR而非IDR。

英文摘要:

We explore two primary classes of approaches to dimensionality reduction (DR): Independent Dimensionality Reduction (IDR) and Simultaneous Dimensionality Reduction (SDR). In IDR methods, of which Principal Components Analysis is a paradigmatic example, each modality is compressed independently, striving to retain as much variation within each modality as possible. In contrast, in SDR, one simultaneously compresses the modalities to maximize the covariation between the reduced descriptions while paying less attention to how much individual variation is preserved. Paradigmatic examples include Partial Least Squares and Canonical Correlations Analysis. Even though these DR methods are a staple of statistics, their relative accuracy and data set size requirements are poorly understood. We introduce a generative linear model to synthesize multimodal data with known variance and covariance structures to examine these questions. We assess the accuracy of the reconstruction of the covariance structures as a function of the number of samples, signal-to-noise ratio, and the number of varying and covarying signals in the data. Using numerical experiments, we demonstrate that linear SDR methods consistently outperform linear IDR methods and yield higher-quality, more succinct reduced-dimensional representations with smaller datasets. Remarkably, regularized CCA can identify low-dimensional weak covarying structures even when the number of samples is much smaller than the dimensionality of the data, which is a regime challenging for all dimensionality reduction methods. Our work corroborates and explains previous observations in the literature that SDR can be more effective in detecting covariation patterns in data. These findings suggest that SDR should be preferred to IDR in real-world data analysis when detecting covariation is more important than preserving variation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑