arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.14608cs.LGstat.ML
收藏

关于UMAP的真损失函数

On UMAP's true loss function

Heidelberg University(海德堡大学)

Sebastian Damrich, Fred A. Hamprecht

摘要:

UMAP已经取代了t-SNE,成为许多学科中可视化高维数据集的最先进技术,但其成功的原因尚不清楚。在这项工作中,我们研究了UMAP的采样为基础的优化方案的详细信息。我们推导了UMAP的有效损失函数的封闭形式,并发现它不同于公布的。因此,我们表明,UMAP并不旨在重现其理论动机的高维UMAP相似性。相反,它试图重现相似性,只编码共享的$k$最近邻图,从而挑战了以前的理解UMAP的有效性。相反,我们认为UMAP成功的关键是它的吸引力和排斥力的隐性平衡所产生的负采样。这种平衡又通过梯度下降促进优化。我们证实了我们的理论研究结果的玩具和单细胞RNA测序数据。

英文摘要:

UMAP has supplanted t-SNE as state-of-the-art for visualizing high-dimensional datasets in many disciplines, but the reason for its success is not well understood. In this work, we investigate UMAP's sampling based optimization scheme in detail. We derive UMAP's effective loss function in closed form and find that it differs from the published one. As a consequence, we show that UMAP does not aim to reproduce its theoretically motivated high-dimensional UMAP similarities. Instead, it tries to reproduce similarities that only encode the shared $k$ nearest neighbor graph, thereby challenging the previous understanding of UMAP's effectiveness. Instead, we claim that the key to UMAP's success is its implicit balancing of attraction and repulsion resulting from negative sampling. This balancing in turn facilitates optimization via gradient descent. We corroborate our theoretical findings on toy and single cell RNA sequencing data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑