arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.05014cs.CLcs.AI
收藏

通过概念瓶颈解释预训练语言模型

Interpreting Pretrained Language Models via Concept Bottlenecks

Arizona State University(亚利桑那州立大学);Zhejiang University(浙江大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校);University of Virginia(弗吉尼亚大学)

Zhen Tan, Lu Cheng, Song Wang, Yuan Bo, Jundong Li, Huan Liu

AI总结:

本研究提出C$^3$M方法,通过结合人工与机器生成的概念瓶颈来解释预训练语言模型,提升可解释性,并帮助诊断失败与增强鲁棒性。

摘要:

预训练语言模型(PLMs)在各种自然语言处理任务中取得了显著进展。然而,由于其“黑箱”特性导致的可解释性缺乏,给负责任地实施带来了挑战。尽管先前的研究尝试通过例如自注意力层中的注意力权重来改善可解释性,但这些权重往往缺乏清晰性、可读性和直观性。在本研究中,我们提出了一种新颖的方法,通过使用人类易于理解的高层次、有意义的文概念来解释预训练语言模型。例如,我们学习“食物”这一概念,并研究它如何影响模型对餐厅评论的情感预测。我们引入了C$^3$M,该方法结合了人工标注和机器生成的概念,以提取旨在封装语义上有意义且任务特定概念的隐藏神经元。通过在真实世界数据集上的实证评估,我们表明我们的方法为解释PLM行为提供了有价值的见解,有助于诊断模型失败,并在嘈杂的概念标签中增强模型的鲁棒性。

英文摘要:

Pretrained language models (PLMs) have made significant strides in various natural language processing tasks. However, the lack of interpretability due to their ``black-box'' nature poses challenges for responsible implementation. Although previous studies have attempted to improve interpretability by using, e.g., attention weights in self-attention layers, these weights often lack clarity, readability, and intuitiveness. In this research, we propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans. For example, we learn the concept of ``Food'' and investigate how it influences the prediction of a model's sentiment towards a restaurant review. We introduce C$^3$M, which combines human-annotated and machine-generated concepts to extract hidden neurons designed to encapsulate semantically meaningful and task-specific concepts. Through empirical evaluations on real-world datasets, we manifest that our approach offers valuable insights to interpret PLM behavior, helps diagnose model failures, and enhances model robustness amidst noisy concept labels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑