意图解释规划：在检索增强语言模型中实现源忠实的解释
Illocutionary Explanation Planning for Source-Faithful Explanations in Retrieval-Augmented Language Models
AI总结：
研究通过意图宏观规划提升检索增强语言模型中解释的源忠实性，通过链式意图提示提升源忠实度，实验显示在部分模型中提升显著，但整体效果仍有限。
摘要：
大型语言模型生成的自然语言解释通常具有说服力，但未必可追溯：用户无法轻易验证解释中的主张是否由证据支持。在XAI中，这促使关注忠实性和可追溯性，即解释主张能否基于明确的来源进行支撑并追溯回来源。我们研究了检索增强生成（RAG）在编程教育中的这些需求，其中教科书提供权威证据。我们对六个LLM在90个基于三个编程教科书的Stack Overflow问题上进行基准测试，并通过来源遵循度量量化来源忠实性。我们发现非检索增强生成（RAG）模型的中位来源遵循度为0%，而基线RAG系统仍表现出低中位遵循度（22-40%，取决于模型）。受Achinstein的解释意图理论启发，我们引入意图宏观规划作为源忠实解释的描述性设计原则，并通过链式意图提示（CoI）实现，将查询扩展为隐式解释问题以驱动检索。在各模型中，CoI在来源遵循度上产生统计显著提升（最高63%），尽管绝对遵循度仍较低，且在某些模型中提升较弱或不显著。一项包含165名保留参与者（220名招募）的用户研究表明，这些提升并未损害满意度、相关性或感知正确性。
英文摘要：
Natural language explanations produced by large language models (LLMs) are often persuasive, but not necessarily scrutable: users cannot easily verify whether the claims in an explanation are supported by evidence. In XAI, this motivates a focus on faithfulness and traceability, i.e., the extent to which an explanation's claims can be grounded in, and traced back to, an explicit source. We study these desiderata in retrieval-augmented generation (RAG) for programming education, where textbooks provide authoritative evidence. We benchmark six LLMs on 90 Stack Overflow questions grounded in three programming textbooks and quantify source faithfulness via source adherence metrics. We find that non Retrieval-Augmented Generation (RAG) models have median source adherence of 0%, while baseline RAG systems still exhibit low median adherence (22-40%, depending on the model). Motivated by Achinstein's illocutionary theory of explanation, we introduce illocutionary macro-planning as a descriptive design principle for source-faithful explanations and instantiate it with chain-of-illocution prompting (CoI), which expands a query into implicit explanatory questions that drive retrieval. Across models, CoI yields statistically significant gains (up to 63%) in source adherence, although absolute adherence remains moderate and the gains are weak or non-significant for some models. A user study with 165 retained participants (220 recruited) indicates that these gains do not harm satisfaction, relevance, or perceived correctness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
