Recent developments in transformer-based language models have allowed them to capture a wide variety of world knowledge that can be adapted to downstream tasks with limited resources. However, what pieces of information are understood in these models is unclear, and neuron-level contributions in identifying them are largely unknown. Conventional approaches in neuron explainability either depend on a finite set of pre-defined descriptors or require manual annotations for training a secondary model that can then explain the neurons of the primary model. In this paper, we take BERT as an example and we try to remove these constraints and propose a novel and scalable framework that ties textual descriptions to neurons. We leverage the potential of generative language models to discover human-interpretable descriptors present in a dataset and use an unsupervised approach to explain neurons with these descriptors. Through various qualitative and quantitative analyses, we demonstrate the effectiveness of this framework in generating useful data-specific descriptors with little human involvement in identifying the neurons that encode these descriptors. In particular, our experiment shows that the proposed approach achieves 75% precision@2, and 50% recall@2
Interpretable Textual Neuron Representations for NLP
Center for Information and Language Processing(信息与语言处理中心);LMU Munich(慕尼黑大学)
输入优化方法,如Google Deep Dream,为计算机视觉DNN创建了可解释的神经元表示。我们提出并评估了将这项技术转移到NLP的方法。我们的研究结果表明,梯度上升与gumbel softmax层产生的n-gram表示,在目标神经元激活方面优于幼稚语料库搜索。这些表征突出了Imaginet架构的语言和视觉模型之间的语法意识差异。
MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation
School of Computer Science and Technology, Tianjin University, Tianjin, China(天津大学计算机科学与技术学院);School of Software, Tsinghua University, Beijing, China(清华大学软件学院);School of Information Resource Management, Renmin University of China,Beijing, China(中国人民大学信息资源管理学院);School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China(上海交通大学人工智能学院);Baidu Inc., Beijing, China(百度公司)