arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.11991cs.LGstat.ML
收藏

通过联合子空间估计从神经网络表示中移除虚假概念

Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimation

University of Amsterdam(阿姆斯特丹大学);Tinbergen Institute(蒂尔堡大学和阿姆斯特丹大学联合的蒂宁根研究所)

Floris Holstege, Bram Wouters, Noud van Giersbergen, Cees Diks

AI总结:

针对神经网络分布外泛化受虚假相关性阻碍及现有概念移除方法易损害主任务性能的问题,提出一种联合估计两个低维正交子空间的迭代算法,有效分离并移除虚假概念,在视觉和NLP基准上优于现有方法。

摘要:

神经网络中的分布外泛化通常受到虚假相关性的阻碍。一种常见策略是通过从数据的神经网络表示中移除虚假概念来缓解此问题。现有的概念移除方法往往过于激进,会无意中消除与模型主任务相关的特征,从而损害模型性能。我们提出了一种迭代算法,通过在神经网络表示中联合识别两个低维正交子空间,将虚假概念与主任务概念分离开来。我们在计算机视觉(Waterbirds、CelebA)和自然语言处理(MultiNLI)的基准数据集上评估了该算法,并表明它优于现有的概念移除方法。

英文摘要:

Out-of-distribution generalization in neural networks is often hampered by spurious correlations. A common strategy is to mitigate this by removing spurious concepts from the neural network representation of the data. Existing concept-removal methods tend to be overzealous by inadvertently eliminating features associated with the main task of the model, thereby harming model performance. We propose an iterative algorithm that separates spurious from main-task concepts by jointly identifying two low-dimensional orthogonal subspaces in the neural network representation. We evaluate the algorithm on benchmark datasets for computer vision (Waterbirds, CelebA) and natural language processing (MultiNLI), and show that it outperforms existing concept removal methods

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑