arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.07697cs.LGcs.CVmath.AT
收藏

用拓扑特征训练的神经网络学习不同的内部表示吗?

Do Neural Networks Trained with Topological Features Learn Different Internal Representations?

Colorado State University(科罗拉多州立大学);University of Texas, El Paso(德克萨斯大学埃尔帕索分校);University of Washington(华盛顿大学);Michigan State University(密歇根州立大学);Pacific Northwest National Laboratory(太平洋西北国家实验室)

Sarah McGuire, Shane Jackson, Tegan Emerson, Henry Kvinge

摘要:

越来越多的工作利用通过拓扑数据分析提取的特征来训练机器学习模型。虽然这一领域,有时被称为拓扑机器学习(TML),已经取得了一些显着的成功,从拓扑特征学习的过程如何不同于从原始数据学习的过程的理解仍然是有限的。在这项工作中,我们开始解决这个更大问题的一个组成部分,询问使用拓扑特征训练的模型是否学习到与使用原始原始数据训练的模型所学习到的数据的内部表示根本不同的数据的内部表示。为了量化"不同“,我们利用了两个流行的度量标准,可以用来衡量神经网络,神经拼接和中心内核对齐内的数据的隐藏表示的相似性。从这些,我们得出了一系列关于拓扑特征训练如何改变和不改变模型学习的表示的结论。也许不足为奇的是,我们发现,在结构上,在拓扑特征上训练和评估的模型的隐藏表示与在相应的原始数据上训练和评估的模型相比有很大的不同。另一方面,我们的实验表明,在某些情况下,这些表示可以使用简单的仿射变换来协调(至少在解决相应任务所需的程度上)。我们推测,这意味着在原始数据上训练的神经网络在进行预测的过程中可能会提取一些有限的拓扑特征。

英文摘要:

There is a growing body of work that leverages features extracted via topological data analysis to train machine learning models. While this field, sometimes known as topological machine learning (TML), has seen some notable successes, an understanding of how the process of learning from topological features differs from the process of learning from raw data is still limited. In this work, we begin to address one component of this larger issue by asking whether a model trained with topological features learns internal representations of data that are fundamentally different than those learned by a model trained with the original raw data. To quantify ``different'', we exploit two popular metrics that can be used to measure the similarity of the hidden representations of data within neural networks, neural stitching and centered kernel alignment. From these we draw a range of conclusions about how training with topological features does and does not change the representations that a model learns. Perhaps unsurprisingly, we find that structurally, the hidden representations of models trained and evaluated on topological features differ substantially compared to those trained and evaluated on the corresponding raw data. On the other hand, our experiments show that in some cases, these representations can be reconciled (at least to the degree required to solve the corresponding task) using a simple affine transformation. We conjecture that this means that neural networks trained on raw data may extract some limited topological features in the process of making predictions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑