揭示语言模型的参数化知识：归因方法的统一框架
Revealing the Parametric Knowledge of Language Models: A Unified Framework for Attribution Methods
AI总结：
本文提出统一评估框架系统比较实例归因(IA)与神经元归因(NA)在揭示语言模型参数化知识上的表现，引入NA-Instances与IA-Neurons对齐方法并设计忠实性测试，发现NA揭示更全面信息而IA提供独特洞察，二者结合有望实现更整体的理解。
摘要：
语言模型（LMs）从其训练过程中获取参数化知识，并将其嵌入在权重中。然而，LMs日益增强的可扩展性给理解模型内部工作机制带来了巨大挑战，也使得在不付出高昂重训练成本的情况下更新或纠正这些嵌入知识变得更加困难。这凸显了确切揭示存储了什么知识及其与特定模型组件之间关联的重要性。实例归因（Instance Attribution, IA）和神经元归因（Neuron Attribution, NA）为这种训练获得的知识提供了洞察，尽管它们尚未被系统地进行比较。我们的研究引入了一种新颖的评估框架，以量化和比较IA和NA所揭示的知识。为了对齐这些方法的结果，我们引入了归因方法NA-Instances，将NA应用于检索有影响力的训练实例；以及IA-Neurons，用于发现由IA发现的有影响力实例的重要神经元。我们进一步提出了一系列全面的忠实性测试，以评估这两种方法提供的解释的全面性和充分性。通过广泛的实验和分析，我们证明，与IA相比，NA通常揭示了关于LM参数化知识的更多样化和更全面的信息。尽管如此，IA提供了关于LM参数化知识的独特且有价值的洞察，而这些是NA未能揭示的。我们的研究结果进一步表明，结合IA和NA的多样化发现，采用协同方法来更全面地理解LM的参数化知识具有潜力。
英文摘要：
Language Models (LMs) acquire parametric knowledge from their training process, embedding it within their weights. The increasing scalability of LMs, however, poses significant challenges for understanding a model's inner workings and further for updating or correcting this embedded knowledge without the significant cost of retraining. This underscores the importance of unveiling exactly what knowledge is stored and its association with specific model components. Instance Attribution (IA) and Neuron Attribution (NA) offer insights into this training-acquired knowledge, though they have not been compared systematically. Our study introduces a novel evaluation framework to quantify and compare the knowledge revealed by IA and NA. To align the results of the methods we introduce the attribution method NA-Instances to apply NA for retrieving influential training instances, and IA-Neurons to discover important neurons of influential instances discovered by IA. We further propose a comprehensive list of faithfulness tests to evaluate the comprehensiveness and sufficiency of the explanations provided by both methods. Through extensive experiments and analysis, we demonstrate that NA generally reveals more diverse and comprehensive information regarding the LM's parametric knowledge compared to IA. Nevertheless, IA provides unique and valuable insights into the LM's parametric knowledge, which are not revealed by NA. Our findings further suggest the potential of a synergistic approach of combining the diverse findings of IA and NA for a more holistic understanding of an LM's parametric knowledge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
