B-cos LM：高效地将预训练语言模型转换以提高可解释性
B-cos LM: Efficiently Transforming Pre-trained Language Models for Improved Explainability
AI总结：
本文提出B-cos LM，通过结合B-cos转换和任务微调，提高预训练语言模型的可解释性与效率。
摘要：
事后解释方法在处理黑盒模型时常常面临忠实性和人类可解释性的问题，这主要是由于当前神经架构缺乏可解释性。同时，B-cos网络已被引入，通过提出一种去除偏差项并促进输入权重对齐的架构来提高模型的可解释性。尽管B-cos网络在构建可解释系统方面已取得成功，但其应用迄今为止仅限于计算机视觉模型及其相关训练流程。在本文中，我们介绍了B-cos LMs，即B-cos语言模型（LMs），用于自然语言处理（NLP）任务。我们的方法通过结合B-cos转换和任务微调，直接将预训练语言模型转换为B-cos LMs，相比之前的方法提高了效率。自动和人工评估结果表明，B-cos LMs产生的解释比事后方法更忠实且更易于人类理解，同时在任务性能上与传统微调相当。我们的深入分析探讨了B-cos LMs在学习过程和解释模式上与传统微调模型的不同之处。最后，我们展示了将解码器-only模型转换为B-cos LMs以用于生成任务的首次探索。我们的代码可在https://github.com/Ewanwong/bcos_lm上获得。
英文摘要：
Post-hoc explanation methods for black-box models often struggle with faithfulness and human interpretability due to the lack of explainability in current neural architectures. Meanwhile, B-cos networks have been introduced to improve model explainability by proposing an architecture that removes bias terms and promotes input-weight alignment. Although B-cos networks have shown success in building explainable systems, their application has so far been limited to computer vision models and their associated training pipelines. In this work, we introduce B-cos LMs, i.e., B-cos Language Models (LMs) empowered for natural language processing (NLP) tasks. Our approach directly transforms pre-trained language models into B-cos LMs by combining B-cos conversion and task fine-tuning, improving efficiency compared to previous methods. Automatic and human evaluation results demonstrate that B-cos LMs produce more faithful and human interpretable explanations than post-hoc methods, while maintaining task performance comparable to conventional fine-tuning. Our in-depth analysis explores how B-cos LMs differ from conventionally fine-tuned models in their learning processes and explanation patterns. Finally, we present a first exploration of transforming decoder-only models to B-cos LMs for generation tasks. Our code is available at https://github.com/Ewanwong/bcos_lm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
