解释语言模型的离散化集成推理
Discretized Integrated Gradients for Explaining Language Models
摘要:
作为一种重要的基于属性的解释算法,综合属性(IG)由于其合理的解释公理和易于梯度计算而被广泛采用。它通过对输入数据空间中沿直线路径内插的模型输出梯度求平均来度量特征重要性。然而,由于词嵌入空间的固有离散性,这种直线内插点不代表文本数据。这就质疑了在插值点处计算的梯度的可靠性,从而质疑了生成的解释的质量。在这里,我们提出了离散化的集成代理(DIG),它允许沿非线性插值路径的有效属性。我们开发了两种插值策略的离散词嵌入空间,生成插值点,位于接近实际的话在嵌入空间中,产生更忠实的梯度计算。我们通过对多个情感分类数据集的实验和人工评估,证明了DIG对IG的有效性。我们提供DIG的源代码,以鼓励可重复的研究。
英文摘要:
As a prominent attribution-based explanation algorithm, Integrated Gradients (IG) is widely adopted due to its desirable explanation axioms and the ease of gradient computation. It measures feature importance by averaging the model's output gradient interpolated along a straight-line path in the input data space. However, such straight-line interpolated points are not representative of text data due to the inherent discreteness of the word embedding space. This questions the faithfulness of the gradients computed at the interpolated points and consequently, the quality of the generated explanations. Here we propose Discretized Integrated Gradients (DIG), which allows effective attribution along non-linear interpolation paths. We develop two interpolation strategies for the discrete word embedding space that generates interpolation points that lie close to actual words in the embedding space, yielding more faithful gradient computation. We demonstrate the effectiveness of DIG over IG through experimental and human evaluations on multiple sentiment classification datasets. We provide the source code of DIG to encourage reproducible research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。