COCKATIEL：通过可解释元素进行连续概念排序归因，用于解释NLP任务上的神经网络分类器
COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP tasks
AI总结：
COCKATIEL是一种基于概念的事后模型无关XAI技术，利用NMF和敏感性分析从Transformer模型末层生成忠实解释，在情感分析任务中无需监督即可发现与人类一致的概念。
摘要：
Transformer架构复杂，其在NLP中的应用虽然带来了许多成功，但也使其可解释性或可解释性变得具有挑战性。近期的争论表明，注意力图和归因方法并不可靠（Pruthi等人，2019；Brunner等人，2019）。在本文中，我们展示了它们的一些局限性，并介绍了COCKATIEL，该方法成功解决了其中的一些问题。COCKATIEL是一种新颖的、事后（post-hoc）的、基于概念的、模型无关的XAI技术，它通过使用非负矩阵分解（NMF）来发现模型在做出预测时所利用的概念，并利用敏感性分析来准确估计每个概念对模型的重要性，从而从针对NLP分类任务训练的神经网络模型的最后一层生成有意义的解释。它这样做不会损害底层模型的准确性，也不需要训练新模型。我们在单方面和多方面情感分析任务中进行了实验，结果表明COCKATIEL在无需任何监督的情况下，在Transformer模型上发现与人类一致的概念方面具有优越能力；我们通过保真度指标客观验证了其解释的忠实性，并展示了其在两个不同数据集上提供有意义解释的能力。
英文摘要：
Transformer architectures are complex and their use in NLP, while it has engendered many successes, makes their interpretability or explainability challenging. Recent debates have shown that attention maps and attribution methods are unreliable (Pruthi et al., 2019; Brunner et al., 2019). In this paper, we present some of their limitations and introduce COCKATIEL, which successfully addresses some of them. COCKATIEL is a novel, post-hoc, concept-based, model-agnostic XAI technique that generates meaningful explanations from the last layer of a neural net model trained on an NLP classification task by using Non-Negative Matrix Factorization (NMF) to discover the concepts the model leverages to make predictions and by exploiting a Sensitivity Analysis to estimate accurately the importance of each of these concepts for the model. It does so without compromising the accuracy of the underlying model or requiring a new one to be trained. We conduct experiments in single and multi-aspect sentiment analysis tasks and we show COCKATIEL's superior ability to discover concepts that align with humans' on Transformer models without any supervision, we objectively verify the faithfulness of its explanations through fidelity metrics, and we showcase its ability to provide meaningful explanations in two different datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
