arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.12191cs.LGcs.CL

核化概念擦除

Kernelized Concept Erasure

Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Bar-Ilan University(巴伊兰大学);University of Cambridge(剑桥大学);ETH Zürich(苏黎世联邦理工学院)

Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, Ryan Cotterell

AI总结:

针对神经网络未必以线性方式编码概念的问题,提出概念擦除线性极小极大博弈的核化方法,可阻止特定非线性敌手预测概念,但保护无法迁移,彻底擦除非线性编码概念仍待解决。

摘要:

文本数据神经模型的表征空间是在训练过程中以无监督方式形成的。理解这些表征如何编码人类可解释的概念是一个基础性问题。识别神经表征中概念的一种主流方法是搜索某个线性子空间,擦除该子空间会阻止从表征中预测概念。然而,尽管许多线性擦除算法易于处理且具有可解释性,但神经网络未必以线性方式表征概念。为识别非线性编码的概念,我们提出了一种用于概念擦除的线性极小极大博弈的核化方法。我们证明可以阻止特定的非线性敌手预测该概念,但这种保护无法迁移到其他不同的非线性敌手。因此,彻底擦除非线性编码的概念仍是一个未解决的问题。

英文摘要:

The representation space of neural models for textual data emerges in an unsupervised manner during training. Understanding how those representations encode human-interpretable concepts is a fundamental problem. One prominent approach for the identification of concepts in neural representations is searching for a linear subspace whose erasure prevents the prediction of the concept from the representations. However, while many linear erasure algorithms are tractable and interpretable, neural networks do not necessarily represent concepts in a linear manner. To identify non-linearly encoded concepts, we propose a kernelization of a linear minimax game for concept erasure. We demonstrate that it is possible to prevent specific non-linear adversaries from predicting the concept. However, the protection does not transfer to different nonlinear adversaries. Therefore, exhaustively erasing a non-linearly encoded concept remains an open problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑