迈向忠实的自然语言解释：一项在大语言模型中使用激活修补的研究
Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models
AI总结：
本研究针对现有大语言模型自然语言解释忠实度衡量方法的缺陷，提出基于激活修补技术的因果忠实度指标，经多规模模型实验验证其更可靠，且对齐微调模型的解释更忠实。
摘要：
大语言模型（LLM）能够生成有说服力的自然语言解释（NLE）来为其答案提供依据。然而，这些解释的忠实度不应仅凭表面就轻易采信。近期研究提出了多种衡量NLE忠实度的方法，通常通过在解释层面或特征层面插入扰动来实现。我们认为，根据已确立的忠实度定义，这些方法既不全面，设计也存在偏差。此外，我们强调了基于分布外样本得出忠实度结论的风险。在本研究中，我们利用一种名为激活修补（activation patching）的因果中介技术，来衡量解释对其所支撑的被解释答案的忠实度。我们提出的指标——因果忠实度（Causal Faithfulness），将解释与对应模型输出之间的因果归因一致性量化为忠实度的衡量指标。我们在参数量从2B到27B不等的多个模型上开展了实验，发现经过对齐微调的模型往往能生成更忠实、更合理的解释。我们发现，因果忠实度相比现有忠实度测试是一项很有前景的改进，它考虑了模型的内部计算过程，并且避免了可能损害忠实度评估有效性的分布外问题。我们在https://github.com/wj210/Causal-Faithfulness发布了相关代码。
英文摘要：
Large Language Models (LLMs) are capable of generating persuasive Natural Language Explanations (NLEs) to justify their answers. However, the faithfulness of these explanations should not be readily trusted at face value. Recent studies have proposed various methods to measure the faithfulness of NLEs, typically by inserting perturbations at the explanation or feature level. We argue that these approaches are neither comprehensive nor correctly designed according to the established definition of faithfulness. Moreover, we highlight the risks of grounding faithfulness findings on out-of-distribution samples. In this work, we leverage a causal mediation technique called activation patching, to measure the faithfulness of an explanation towards supporting the explained answer. Our proposed metric, Causal Faithfulness quantifies the consistency of causal attributions between explanations and the corresponding model outputs as the indicator of faithfulness. We experimented across models varying from 2B to 27B parameters and found that models that underwent alignment tuning tend to produce more faithful and plausible explanations. We find that Causal Faithfulness is a promising improvement over existing faithfulness tests by taking into account the model's internal computations and avoiding out of distribution concerns that could otherwise undermine the validity of faithfulness assessments. We release the code in \url{https://github.com/wj210/Causal-Faithfulness}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
