arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.17154cs.LGcs.AI
收藏

深度网络表示中的凸决策区域研究

On convex decision regions in deep network representations

Section for Cognitive Systems, DTU Compute, Technical University of Denmark(认知系统部门,DTU计算学院,丹麦技术大学)

Lenka Tětková, Thea Brüsch, Teresa Karen Scheidt, Fabian Martin Mager, Rasmus Ørtoft Aagaard, Jonathan Foldager, Tommy Sonne Alstrøm, Lars Kai Hansen

AI总结:

本文研究深度网络潜在空间中概念区域的凸性,开发测量工具并发现近似凸性普遍存在,且预训练凸性可预测微调性能。

摘要:

当前关于人机对齐的研究旨在理解机器学习得到的潜在空间及其与人类表征的对应关系。Gärdenfors的概念空间是理解人类表征的一个重要框架。概念空间中对象区域的凸性被认为有助于提升泛化能力、少样本学习和人际对齐。基于这些见解,我们研究了机器学习潜在空间中概念区域的凸性概念。我们开发了一套用于在采样数据中测量凸性的工具,并评估了最先进深度网络的分层表示中涌现的凸性。我们表明凸性对基本重参数化具有鲁棒性,因此作为机器学习潜在空间的一种性质是有意义的。我们发现近似凸性在多个应用领域的神经表示中普遍存在,包括图像、音频、人类活动、文本和医学图像模型。总体而言,我们观察到微调会增加标签区域的凸性。我们发现证据表明,类别标签区域的预训练凸性可以预测后续微调的性能。

英文摘要:

Current work on human-machine alignment aims at understanding machine-learned latent spaces and their correspondence to human representations. G{ä}rdenfors' conceptual spaces is a prominent framework for understanding human representations. Convexity of object regions in conceptual spaces is argued to promote generalizability, few-shot learning, and interpersonal alignment. Based on these insights, we investigate the notion of convexity of concept regions in machine-learned latent spaces. We develop a set of tools for measuring convexity in sampled data and evaluate emergent convexity in layered representations of state-of-the-art deep networks. We show that convexity is robust to basic re-parametrization and, hence, meaningful as a quality of machine-learned latent spaces. We find that approximate convexity is pervasive in neural representations in multiple application domains, including models of images, audio, human activity, text, and medical images. Generally, we observe that fine-tuning increases the convexity of label regions. We find evidence that pretraining convexity of class label regions predicts subsequent fine-tuning performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑