说到做到？衡量大语言模型解释的忠实度
Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations
AI总结：
针对大语言模型解释可能不忠实的问题，提出基于概念差异的忠实度定义，结合辅助LLM生成反事实与贝叶斯层次模型的估算方法，可量化并发现不忠实模式。
摘要：
大语言模型（LLM）能够为其如何得出问题答案生成看似合理的解释。然而，这些解释可能会歪曲模型的“推理”过程，也就是说它们可能是不忠实的。这反过来又会导致过度信任和误用。我们提出了一种衡量LLM解释忠实度的新方法。首先，我们给出了忠实度的严格定义。由于LLM的解释模仿人类解释，它们通常会引用输入问题中据称影响了模型的高层概念。我们将忠实度定义为：LLM解释所暗示的具有影响力的概念集合，与真正具有影响力的概念集合之间的差异。其次，我们提出了一种估算忠实度的新方法，该方法基于：（1）使用辅助LLM修改模型输入中的概念值，生成符合现实的反事实样本；（2）使用贝叶斯层次模型在样本层面和数据集层面量化概念的因果效应。我们的实验表明，该方法可用于量化并发现可解释的不忠实模式。在社会偏见任务中，我们发现了LLM解释掩盖社会偏见影响的案例；在医疗问答任务中，我们发现了LLM解释就哪些证据影响了模型决策给出误导性说法的案例。
英文摘要：
Large language models (LLMs) are capable of generating plausible explanations of how they arrived at an answer to a question. However, these explanations can misrepresent the model's "reasoning" process, i.e., they can be unfaithful. This, in turn, can lead to over-trust and misuse. We introduce a new approach for measuring the faithfulness of LLM explanations. First, we provide a rigorous definition of faithfulness. Since LLM explanations mimic human explanations, they often reference high-level concepts in the input question that purportedly influenced the model. We define faithfulness in terms of the difference between the set of concepts that LLM explanations imply are influential and the set that truly are. Second, we present a novel method for estimating faithfulness that is based on: (1) using an auxiliary LLM to modify the values of concepts within model inputs to create realistic counterfactuals, and (2) using a Bayesian hierarchical model to quantify the causal effects of concepts at both the example- and dataset-level. Our experiments show that our method can be used to quantify and discover interpretable patterns of unfaithfulness. On a social bias task, we uncover cases where LLM explanations hide the influence of social bias. On a medical question answering task, we uncover cases where LLM explanations provide misleading claims about which pieces of evidence influenced the model's decisions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
