arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.10795cs.CL
收藏

针对合理化模型解释鲁棒性的对抗攻击

Adversarial Attack for Explanation Robustness of Rationalization Models

School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);iWudao Tech(悟道科技)

Yuankai Zhang, Lingxiao Kong, Haozhao Wang, Ruixuan Li, Jun Wang, Yuhua Li, Wei Liu

AI总结:

本文提出UAT2E对抗攻击方法,在不改变预测的情况下削弱合理化模型的可解释性,实验揭示其脆弱性并提出改进建议。

摘要:

合理化模型选择输入文本的子集作为基本原理,这对于人类理解和信任预测至关重要,最近已成为可解释人工智能领域的一个突出研究方向。然而,以往的大多数研究主要集中于提高基本原理的质量,而忽略了其对恶意攻击的鲁棒性。具体而言,在对抗攻击下,合理化模型是否仍能生成高质量的基本原理尚不清楚。为了探究这一问题,本文提出了UAT2E,旨在削弱合理化模型的可解释性而不改变其预测,从而引发人类用户对这些模型的不信任。UAT2E采用基于梯度的触发器搜索,然后将触发器插入原始输入中,以进行非目标和目标攻击。在五个数据集上的实验结果揭示了合理化模型在解释方面的脆弱性,即在攻击下它们倾向于选择更多无意义的标记。基于此,我们提出了一系列改进合理化模型解释能力的建议。

英文摘要:

Rationalization models, which select a subset of input text as rationale-crucial for humans to understand and trust predictions-have recently emerged as a prominent research area in eXplainable Artificial Intelligence. However, most of previous studies mainly focus on improving the quality of the rationale, ignoring its robustness to malicious attack. Specifically, whether the rationalization models can still generate high-quality rationale under the adversarial attack remains unknown. To explore this, this paper proposes UAT2E, which aims to undermine the explainability of rationalization models without altering their predictions, thereby eliciting distrust in these models from human users. UAT2E employs the gradient-based search on triggers and then inserts them into the original input to conduct both the non-target and target attack. Experimental results on five datasets reveal the vulnerability of rationalization models in terms of explanation, where they tend to select more meaningless tokens under attacks. Based on this, we make a series of recommendations for improving rationalization models in terms of explanation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑