抽取式自我解释与人类推理在文本分类中的系统比较
A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification
AI总结：
本文比较了抽取式自我解释与人类推理在文本分类任务中的有效性，通过分析不同任务和语言的解释质量，发现自我解释在文本长度和任务复杂度上与人类推理存在显著差异。
摘要：
指令微调的LLM能够通过生成自我解释来向用户解释其输出，而无需应用复杂的可解释性技术。本文分析这种能力是否能产生高质量的解释。我们评估了以输入推理形式呈现的自我解释在人类中的可信度。我们研究了三个文本分类任务：情感分类、强迫劳动检测和声明验证。我们包括丹麦语和意大利语的情感分类任务翻译，并将自我解释与人类注释进行比较。为此，我们收集了Climate-Fever声明验证数据集的人类推理注释。我们进一步评估了人类和自我解释推理在正确模型预测方面的忠实度，并通过纳入事后归因基于的解释扩展了研究。我们分析了四个开源LLM，并发现自我解释与人类推理之间的对齐高度依赖于文本长度和任务复杂性。然而，自我解释会产生忠实的token级推理子集，而事后归因方法则倾向于强调结构和格式token，反映出根本不同的解释策略。
英文摘要：
Instruction-tuned LLMs are able to provide \textit{an} explanation about their output to users by generating self-explanations, without requiring the application of complex interpretability techniques. In this paper, we analyse whether this ability results in a \textit{good} explanation. We evaluate self-explanations in the form of input rationales with respect to their plausibility to humans. We study three text classification tasks: sentiment classification, forced labour detection and claim verification. We include Danish and Italian translations of the sentiment classification task and compare self-explanations to human annotations. For this, we collected human rationale annotations for Climate-Fever, a claim verification dataset. We furthermore evaluate the faithfulness of human and self-explanation rationales with respect to correct model predictions, and extend the study by incorporating post-hoc attribution-based explanations. We analyse four open-weight LLMs and find that alignment between self-explanations and human rationales highly depends on text length and task complexity. Nevertheless, self-explanations yield faithful subsets of token-level rationales, whereas post-hoc attribution methods tend to emphasize structural and formatting tokens, reflecting fundamentally different explanation strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
