arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.16731cs.CLcs.AI
收藏

面向生成语言模型中神经元的信息性文本描述

Towards Generating Informative Textual Description for Neurons in Language Models

UMass Amherst(马萨诸塞大学阿默斯特分校);IBM Research(IBM研究院)

Shrayani Mondal, Rishabh Garodia, Arbaaz Qureshi, Taesung Lee, Youngja Park

AI总结:

针对语言模型神经元可解释性方法依赖预定义描述符或人工标注的问题,研究提出将文本描述与神经元关联的无监督可扩展框架,以BERT为例验证了其生成描述符的有效性,precision@2达75%、recall@2达50%。

摘要:

基于Transformer的语言模型的最新发展使其能够捕获广泛的世界知识,可在资源有限的情况下适配下游任务。然而,这些模型理解哪些信息尚不明确,且神经元在识别这些信息中的作用在很大程度上仍未可知。传统的神经元可解释性方法要么依赖于有限的预定义描述符集,要么需要人工标注来训练次级模型,再用该模型解释主模型的神经元。本文以BERT为例,尝试消除这些限制,提出了一种新颖且可扩展的框架,将文本描述与神经元关联起来。我们利用生成式语言模型的潜力,发现数据集中存在的人类可解释描述符,并采用无监督方法用这些描述符解释神经元。通过多项定性和定量分析,我们证明了该框架在生成有用的特定数据描述符方面的有效性,且在识别编码这些描述符的神经元时几乎不需要人工参与。具体而言,实验表明所提方法的precision@2达到75%,recall@2达到50%。

英文摘要:

Recent developments in transformer-based language models have allowed them to capture a wide variety of world knowledge that can be adapted to downstream tasks with limited resources. However, what pieces of information are understood in these models is unclear, and neuron-level contributions in identifying them are largely unknown. Conventional approaches in neuron explainability either depend on a finite set of pre-defined descriptors or require manual annotations for training a secondary model that can then explain the neurons of the primary model. In this paper, we take BERT as an example and we try to remove these constraints and propose a novel and scalable framework that ties textual descriptions to neurons. We leverage the potential of generative language models to discover human-interpretable descriptors present in a dataset and use an unsupervised approach to explain neurons with these descriptors. Through various qualitative and quantitative analyses, we demonstrate the effectiveness of this framework in generating useful data-specific descriptors with little human involvement in identifying the neurons that encode these descriptors. In particular, our experiment shows that the proposed approach achieves 75% precision@2, and 50% recall@2

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑