arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.02068cs.CL
收藏

利用解释方法进行模型增强

Using Interpretation Methods for Model Enhancement

School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Shanghai Engineering Research Center of Intelligent Vision and Imaging(上海智能视觉与成像工程技术研究中心)

Zhuo Chen, Chengyue Jiang, Kewei Tu

AI总结:

本文提出一个利用解释方法与黄金依据增强神经模型的通用框架,并基于擦除/替换和提取器方法设计两种新实例,在低资源场景下显著优于梯度方法。

摘要:

在神经自然语言处理时代,有大量工作试图推导神经模型的解释。直观上,当训练过程中存在黄金依据时,可以额外训练模型使其解释与黄金依据相匹配。然而,这一直观想法尚未被充分探索。在本文中,我们提出了一个利用解释方法和黄金依据来增强模型的框架。我们的框架非常通用,因为它可以纳入各种解释方法。先前提出的基于梯度的方法可以被证明是我们框架的一个实例。我们还提出了两个新颖的实例,分别利用另外两类解释方法,即基于擦除/替换的方法和基于提取器的方法,用于模型增强。我们在多种任务上进行了全面的实验。实验结果表明,我们的框架在低资源环境下利用各种解释方法增强模型尤为有效,并且我们新提出的两种方法在大多数环境下优于基于梯度的方法。代码可在 https://github.com/Chord-Chen-30/UIMER 获取。

英文摘要:

In the age of neural natural language processing, there are plenty of works trying to derive interpretations of neural models. Intuitively, when gold rationales exist during training, one can additionally train the model to match its interpretation with the rationales. However, this intuitive idea has not been fully explored. In this paper, we propose a framework of utilizing interpretation methods and gold rationales to enhance models. Our framework is very general in the sense that it can incorporate various interpretation methods. Previously proposed gradient-based methods can be shown as an instance of our framework. We also propose two novel instances utilizing two other types of interpretation methods, erasure/replace-based and extractor-based methods, for model enhancement. We conduct comprehensive experiments on a variety of tasks. Experimental results show that our framework is effective especially in low-resource settings in enhancing models with various interpretation methods, and our two newly-proposed methods outperform gradient-based methods in most settings. Code is available at https://github.com/Chord-Chen-30/UIMER.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑