arXiv 2402.046782025-10-27cs.CLcs.AIcs.LG

FaithLM：迈向大语言模型的忠实解释

FaithLM: Towards Faithful Explanations for Large Language Models

Rice University（里士大学）；Texas A&M University（德克萨斯农工大学）；Wake Forest University（沃杰森林大学）；New Jersey Institute of Technology（新泽西理工学院）；John Hopkins University（约翰霍普金斯大学）；Google Research（谷歌研究）；Meta Platforms, Inc.（元宇宙平台公司）

Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Ruixiang Tang, Shaochen Zhong, Fan Yang, Mengnan Du, Xuanting Cai, Vladimir Braverman, Xia Hu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

FaithLM通过干预属性评估并迭代优化大语言模型的解释忠实性，在多个数据集上显著提升解释与人类理据的一致性。

摘要：

大语言模型（LLMs）越来越多地生成自然语言解释，然而这些解释往往缺乏忠实性，且不能可靠地反映模型用于决策的证据。我们提出了FaithLM，一个模型无关的框架，用于评估和改进LLM解释的忠实性，无需令牌掩蔽或任务特定的启发式方法。FaithLM将解释忠实性形式化为一种干预属性：当忠实解释的内容被反驳时，应产生预测偏移。理论分析表明，由此产生的相反提示分数是忠实性的一个健全且具有区分度的估计量。基于这一原则，FaithLM迭代地优化提示词和解释，以最大化测得的分数。在三个多领域数据集和多个LLM骨干上的实验表明，FaithLM持续提高忠实性，并生成比强自解释基线更符合人类理据的解释。这些发现强调，基于干预的评估结合迭代优化，为获得忠实且可靠的LLM解释提供了一条原则性路径。

英文摘要：

Large language models (LLMs) increasingly produce natural language explanations, yet these explanations often lack faithfulness, and they do not reliably reflect the evidence the model uses to decide. We introduce FaithLM, a model-agnostic framework that evaluates and improves the faithfulness of LLM explanations without token masking or task-specific heuristics. FaithLM formalizes explanation faithfulness as an intervention property: a faithful explanation should yield a prediction shift when its content is contradicted. Theoretical analysis shows that the resulting contrary-hint score is a sound and discriminative estimator of faithfulness. Building on this principle, FaithLM iteratively refines both the elicitation prompt and the explanation to maximize the measured score. Experiments on three multi-domain datasets and multiple LLM backbones demonstrate that FaithLM consistently increases faithfulness and produces explanations more aligned with human rationales than strong self-explanation baselines. These findings highlight that intervention-based evaluation, coupled with iterative optimization, provides a principled route toward faithful and reliable LLM explanations.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文