arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.02433stat.MLcs.LGmath.AP
收藏

关于最优降维中的概率嵌入

On Probabilistic Embeddings in Optimal Dimension Reduction

North Carolina State University(北卡罗来纳州立大学)

Ryan Murray, Adam Pickarski

AI总结:

本文研究广义多维尺度分析降维问题的变分性质,证明松弛概率形式的全局最优解必为确定性嵌入,并揭示标准计算方法学到次优映射及误导性聚类结构。

摘要:

降维算法是许多数据科学流程中的关键组成部分,包括数据探索、特征创建与选择以及去噪。尽管它们被广泛使用,许多非线性降维算法在理论层面的理解仍然不足。本文考虑多维尺度分析的一种广义版本,该版本被表述为一个优化问题:从高维特征空间到低维嵌入空间的映射试图保持特征空间中分布的内积或范数,并且涵盖了许多常用的降维算法。我们对该问题的变分性质进行了分析研究,得到以下见解:1)使用标准粒子下降方法求得的解可能导致非确定性嵌入;2)该问题的松弛或概率化形式允许得到具有易于解释的必要条件的解;3)松弛问题的全局最优解实际上必须给出确定性嵌入。这一系列结果与最优传输的经典发展历程相呼应,并且在与Gromov-Wasserstein距离相关的情形中,实际上明确揭示了最优嵌入的结构,这些嵌入由参数确定且不连续。最后,我们说明该任务的一种标准计算实现并未学到确定性嵌入,这意味着它学到的是次优映射,并且在该情境下学到的嵌入具有高度误导性的聚类结构,凸显了在计算上求解该问题的微妙本质。

英文摘要:

Dimension reduction algorithms are a crucial part of many data science pipelines, including data exploration, feature creation and selection, and denoising. Despite their wide utilization, many non-linear dimension reduction algorithms are poorly understood from a theoretical perspective. In this work we consider a generalized version of multidimensional scaling, which is posed as an optimization problem in which a mapping from a high-dimensional feature space to a lower-dimensional embedding space seeks to preserve either inner products or norms of the distribution in feature space, and which encompasses many commonly used dimension reduction algorithms. We analytically investigate the variational properties of this problem, leading to the following insights: 1) Solutions found using standard particle descent methods may lead to non-deterministic embeddings, 2) A relaxed or probabilistic formulation of the problem admits solutions with easily interpretable necessary conditions, 3) The globally optimal solutions to the relaxed problem actually must give a deterministic embedding. This progression of results mirrors the classical development of optimal transportation, and in a case relating to the Gromov-Wasserstein distance actually gives explicit insight into the structure of the optimal embeddings, which are parametrically determined and discontinuous. Finally, we illustrate that a standard computational implementation of this task does not learn deterministic embeddings, which means that it learns sub-optimal mappings, and that the embeddings learned in that context have highly misleading clustering structure, underscoring the delicate nature of solving this problem computationally.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑