DecompX:通过传播 token 分解来解释 Transformer 的决策
DecompX: Explaining Transformers Decisions by Propagating Token Decomposition
AI总结:
本文提出 DecompX,通过在 Transformer 各层传播未混合的分解 token 表征并纳入分类头,生成与预测对应的忠实解释,在多数据集上优于梯度和向量方法。
摘要:
解释基于 Transformer 的模型的一种新兴解决方案,是对表征如何形成进行基于向量的分析。然而,为多层模型提供忠实的基于向量的解释在三个方面可能具有挑战性:(1)将所有组件纳入分析;(2)聚合各层动态,以确定整个模型中的信息流与混合;(3)识别基于向量的分析与模型预测之间的联系。本文提出 DecompX 来应对这些挑战。DecompX 基于构建分解后的 token 表征,并在模型中连续传播这些表征,而不在层与层之间将其混合。此外,与现有解决方案相比,我们的方案具有多项优势,因为它纳入了所有编码器组件(尤其是非线性前馈网络)以及分类头。前者允许获得精确向量,而后者将分解转化为有意义的、基于预测的值,从而无需基于范数或求和的向量聚合。根据标准忠实度评估,DecompX 在多个数据集上始终优于现有基于梯度和基于向量的方法。我们的代码可在 https://github.com/mohsenfayyaz/DecompX 获取。
英文摘要:
An emerging solution for explaining Transformer-based models is to use vector-based analysis on how the representations are formed. However, providing a faithful vector-based explanation for a multi-layer model could be challenging in three aspects: (1) Incorporating all components into the analysis, (2) Aggregating the layer dynamics to determine the information flow and mixture throughout the entire model, and (3) Identifying the connection between the vector-based analysis and the model's predictions. In this paper, we present DecompX to tackle these challenges. DecompX is based on the construction of decomposed token representations and their successive propagation throughout the model without mixing them in between layers. Additionally, our proposal provides multiple advantages over existing solutions for its inclusion of all encoder components (especially nonlinear feed-forward networks) and the classification head. The former allows acquiring precise vectors while the latter transforms the decomposition into meaningful prediction-based values, eliminating the need for norm- or summation-based vector aggregation. According to the standard faithfulness evaluations, DecompX consistently outperforms existing gradient-based and vector-based approaches on various datasets. Our code is available at https://github.com/mohsenfayyaz/DecompX.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。