arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.01514cs.LGmath.ATstat.ML
收藏

映射潜表示的多重宇宙

Mapping the Multiverse of Latent Representations

Helmholtz Munich(亥姆霍兹慕尼黑中心);Technical University of Munich(慕尼黑工业大学);KTH Royal Institute of Technology(皇家理工学院(瑞典));Max Planck Institute for Informatics(马克斯·普朗克信息学研究所)

Jeremy Wayland, Corinna Coupette, Bastian Rieck

AI总结:

针对机器学习潜表示变异性导致的复杂性与不可信问题,提出基于持久同调的框架PRESTO,用于映射和比较不同模型、参数与数据集组合下的潜空间,支持敏感性分析、异常嵌入检测及超参数搜索。

摘要:

为响应近期通过多重宇宙分析来应对机器学习中可靠性与鲁棒性问题的呼吁,我们提出了PRESTO,一个用于映射依赖潜表示的机器学习模型多重宇宙的原则性框架。尽管此类模型被广泛采用,其嵌入的变异性仍鲜为人知,导致了不必要的复杂性与不可信的表示。我们的框架使用持久同调来刻画由不同机器学习方法、(超)参数配置和数据集组合产生的潜空间,使我们能够测量它们的两两(不)相似性,并对其分布进行统计推理。正如我们在理论与实证上所证明,我们的流水线保留了潜表示集合的理想性质,并可用于执行敏感性分析、检测异常嵌入,或高效且有效地导航超参数搜索空间。

英文摘要:

Echoing recent calls to counter reliability and robustness concerns in machine learning via multiverse analysis, we present PRESTO, a principled framework for mapping the multiverse of machine-learning models that rely on latent representations. Although such models enjoy widespread adoption, the variability in their embeddings remains poorly understood, resulting in unnecessary complexity and untrustworthy representations. Our framework uses persistent homology to characterize the latent spaces arising from different combinations of diverse machine-learning methods, (hyper)parameter configurations, and datasets, allowing us to measure their pairwise (dis)similarity and statistically reason about their distributions. As we demonstrate both theoretically and empirically, our pipeline preserves desirable properties of collections of latent representations, and it can be leveraged to perform sensitivity analysis, detect anomalous embeddings, or efficiently and effectively navigate hyperparameter search spaces.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑