arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.03149cs.LGcs.AIcs.CV
收藏

将神经网络中学习到的概念归因于训练数据

Attributing Learned Concepts in Neural Networks to Training Data

Pacific Northwest National Laboratory(太平洋西北国家实验室);Duke University(杜克大学);Thompson Reuters Labs(汤森路透实验室);University of Washington(华盛顿大学)

Nicholas Konz, Charles Godfrey, Madelyn Shapiro, Jonathan Tu, Henry Kvinge, Davis Brown

AI总结:

本文结合数据归因与概念探测方法,利用TRAK分析训练数据对神经网络概念学习的影响,发现概念形成具有分散性与鲁棒性。

摘要:

迄今为止,已有大量证据表明,深度学习模型在其内部数据表征中会学习到某些人类可解释的特征。由于拥有正确(或错误)的概念对于可信的机器学习系统至关重要,因此自然会产生一个问题:在模型原始训练集中,哪些输入对于在特定层学习某个概念最为重要?为了回答这一问题,我们将数据归因方法与探测模型所学概念的方法相结合。针对两个概念数据集,在多个网络层上训练网络和探测集成,我们使用了近期开发的TRAK方法进行大规模数据归因。我们找到了一些收敛性证据:移除某个概念的10,000个最高归因图像并重新训练模型,并不会改变该概念在网络中的位置,也不会改变该概念的探测稀疏性。这表明,并非高度依赖于少数特定示例,而是告知概念发展的特征在其样本中以更分散的方式分布,从而暗示了概念形成中的鲁棒性。

英文摘要:

By now there is substantial evidence that deep learning models learn certain human-interpretable features as part of their internal representations of data. As having the right (or wrong) concepts is critical to trustworthy machine learning systems, it is natural to ask which inputs from the model's original training set were most important for learning a concept at a given layer. To answer this, we combine data attribution methods with methods for probing the concepts learned by a model. Training network and probe ensembles for two concept datasets on a range of network layers, we use the recently developed TRAK method for large-scale data attribution. We find some evidence for convergence, where removing the 10,000 top attributing images for a concept and retraining the model does not change the location of the concept in the network nor the probing sparsity of the concept. This suggests that rather than being highly dependent on a few specific examples, the features that inform the development of a concept are spread in a more diffuse manner across its exemplars, implying robustness in concept formation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑