GAProtoNet:一种基于多头图注意力的原型网络,用于可解释文本分类
GAProtoNet: A Multi-head Graph Attention-based Prototypical Network for Interpretable Text Classification
AI总结:
本文提出GAProtoNet,一种基于多头图注意力的白盒原型网络,通过将输入和原型视为图节点学习可解释表示,在不牺牲准确率的前提下实现文本分类决策的透明解释。
摘要:
基于预训练Transformer的语言模型(LMs)以其强大的词嵌入能力在文本分类任务上取得显著提升而闻名,但其黑盒特性导致的可解释性缺失一直是个主要问题。在本研究中,我们提出了GAProtoNet,一种新型白盒多头图注意力原型网络,旨在解释基于LM编码器构建的文本分类模型的决策。在我们的方法中,输入向量和原型被视为图中的节点,我们利用多头图注意力选择性地在输入节点和原型节点之间构建边,以学习可解释的原型表示。在推理过程中,模型基于激活原型的线性组合做出决策,该组合由分配给每个原型的注意力分数加权,使其选择能够通过注意力权重和投影到最匹配训练样本中的原型透明地解释。在多个公开数据集上的实验表明,我们的方法在不牺牲原始黑盒LMs准确率的情况下取得了优异结果。我们还与四种替代原型网络变体进行了比较,我们的方法在所有方法中实现了最佳准确率和F1分数。我们的案例研究和原型聚类可视化也证明了其在解释基于LMs构建的黑盒模型决策方面的有效性。
英文摘要:
Pretrained transformer-based Language Models (LMs) are well-known for their ability to achieve significant improvement on text classification tasks with their powerful word embeddings, but their black-box nature, which leads to a lack of interpretability, has been a major concern. In this work, we introduce GAProtoNet, a novel white-box Multi-head Graph Attention-based Prototypical Network designed to explain the decisions of text classification models built with LM encoders. In our approach, the input vector and prototypes are regarded as nodes within a graph, and we utilize multi-head graph attention to selectively construct edges between the input node and prototype nodes to learn an interpretable prototypical representation. During inference, the model makes decisions based on a linear combination of activated prototypes weighted by the attention score assigned for each prototype, allowing its choices to be transparently explained by the attention weights and the prototypes projected into the closest matching training examples. Experiments on multiple public datasets show our approach achieves superior results without sacrificing the accuracy of the original black-box LMs. We also compare with four alternative prototypical network variations and our approach achieves the best accuracy and F1 among all. Our case study and visualization of prototype clusters also demonstrate the efficiency in explaining the decisions of black-box models built with LMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。