LIREx:用相关解释增强语言推理
LIREx: Augmenting Language Inference with Relevant Explanation
摘要:
自然语言解释(NLE)是一种特殊形式的数据注释,其中注释者在为数据实例分配标签时识别基本原理(最重要的文本标记),并基于基本原理用自然语言写出对标签的解释。NLE已被证明可以更好地捕捉人类推理,但对于自然语言推理(NLI)来说并不那么有益。在本文中,我们分析了两个主要缺陷的方式NLE目前用于训练解释生成器的语言推理任务。我们发现,解释发电机不考虑到固有的可变性,在人类的解释标签,目前的解释生成模型产生虚假的解释。为了克服这些限制,我们提出了一个新的框架,LIREx,它结合了一个合理的解释生成器和一个实例选择器,只选择相关的,合理的NLE来增强NLI模型。在标准化SNLI数据集上进行评估时,LIREx的准确率达到91.87%,比基线提高了0.32,与数据集上报告的最佳性能相匹配。当转移到域外MultiNLI数据集时,它也实现了比以前的研究更好的性能。定性分析表明,LIREx生成灵活,忠实和相关的NLE,使模型更强大的虚假解释。该代码可在https://github.com/zhaoxy92/LIREx上获得。
英文摘要:
Natural language explanations (NLEs) are a special form of data annotation in which annotators identify rationales (most significant text tokens) when assigning labels to data instances, and write out explanations for the labels in natural language based on the rationales. NLEs have been shown to capture human reasoning better, but not as beneficial for natural language inference (NLI). In this paper, we analyze two primary flaws in the way NLEs are currently used to train explanation generators for language inference tasks. We find that the explanation generators do not take into account the variability inherent in human explanation of labels, and that the current explanation generation models generate spurious explanations. To overcome these limitations, we propose a novel framework, LIREx, that incorporates both a rationale-enabled explanation generator and an instance selector to select only relevant, plausible NLEs to augment NLI models. When evaluated on the standardized SNLI data set, LIREx achieved an accuracy of 91.87%, an improvement of 0.32 over the baseline and matching the best-reported performance on the data set. It also achieves significantly better performance than previous studies when transferred to the out-of-domain MultiNLI data set. Qualitative analysis shows that LIREx generates flexible, faithful, and relevant NLEs that allow the model to be more robust to spurious explanations. The code is available at https://github.com/zhaoxy92/LIREx.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。