Explaining Pre-Trained Language Models with Attribution Scores: An Analysis in Low-Resource Settings
Bosch Center for Artificial Intelligence(博世人工智能中心);University of Stuttgart(斯图加特大学);Hochschule der Medien(媒体大学);Technical University of Darmstadt(达姆施塔特工业大学)
Wei Zhou, Heike Adel, Hendrik Schuff, Ngoc Thang Vu
归因分数可指示不同输入部分的重要性,因此能够解释模型行为。当前,基于提示的模型正日益流行,部分原因是其在低资源场景下更易适配。然而,从基于提示的模型中提取的归因分数的质量尚未得到研究。在本研究中,我们围绕这一主题,分析从基于提示的模型中提取的归因分数的合理性与忠实性,并将其与从微调模型及大语言模型中提取的归因分数进行对比。与以往研究不同,我们将训练规模作为另一个维度纳入分析。我们发现,在低资源场景下,使用提示范式(无论是基于编码器还是基于解码器的模型)都能产生比微调模型更合理的解释;并且Shapley Value Sampling(沙普利值采样)在生成更合理、更忠实的解释方面,始终优于注意力机制和Integrated Gradients(积分梯度)。
英文摘要:
Attribution scores indicate the importance of different input parts and can, thus, explain model behaviour. Currently, prompt-based models are gaining popularity, i.a., due to their easier adaptability in low-resource settings. However, the quality of attribution scores extracted from prompt-based models has not been investigated yet. In this work, we address this topic by analyzing attribution scores extracted from prompt-based models w.r.t. plausibility and faithfulness and comparing them with attribution scores extracted from fine-tuned models and large language models. In contrast to previous work, we introduce training size as another dimension into the analysis. We find that using the prompting paradigm (with either encoder-based or decoder-based models) yields more plausible explanations than fine-tuning the models in low-resource settings and Shapley Value Sampling consistently outperforms attention and Integrated Gradients in terms of leading to more plausible and faithful explanations.
A Novel Interpretability Metric for Explaining Bias in Language Models: Applications on Multilingual Models from Southeast Asia
School of Computer Science, University of Birmingham(英国伯明翰大学计算机科学学院);Department of Information Systems and Computer Science, Ateneo de Manila University(马尼拉大学信息系统与计算机科学系)
Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Framework
Tencent(腾讯);Tsinghua University(清华大学);University of the Chinese Academy of Sciences(中国科学院大学);Huazhong University of Science and Technology(华中科技大学);Guangdong University of Foreign Studies(广东外语外贸大学);Southeast University(东南大学)
Explaining Black-box Model Predictions via Two-level Nested Feature Attributions with Consistency Property
STAIR Lab, Chiba Institute of Technology(千叶技术学院STAIR实验室);School of Mathematics and Statistics, The University of Melbourne(墨尔本大学数学与统计学学院);ZOZO Research(ZOZO研究)