Many methods now exist for conditioning model outputs on task instructions, retrieved documents, and user-provided explanations and feedback. Rather than relying solely on examples of task inputs and outputs, these approaches use valuable additional data for improving model correctness and aligning learned models with human priors. Meanwhile, a growing body of evidence suggests that some language models can (1) store a large amount of knowledge in their parameters, and (2) perform inference over tasks in textual inputs at test time. These results raise the possibility that, for some tasks, humans cannot explain to a model any more about the task than it already knows or could infer on its own. In this paper, we study the circumstances under which explanations of individual data points can (or cannot) improve modeling performance. In order to carefully control important properties of the data and explanations, we introduce a synthetic dataset for experiments, and we also make use of three existing datasets with explanations: e-SNLI, TACRED, and SemEval. We first give a formal framework for the available modeling approaches, in which explanation data can be used as model inputs, as targets, or as a prior. After arguing that the most promising role for explanation data is as model inputs, we propose to use a retrieval-based method and show that it solves our synthetic task with accuracies upwards of 95%, while baselines without explanation data achieve below 65% accuracy. We then identify properties of datasets for which retrieval-based modeling fails. With the three existing datasets, we find no improvements from explanation retrieval. Drawing on findings from our synthetic task, we suggest that at least one of six preconditions for successful modeling fails to hold with these datasets. Our code is publicly available at https://github.com/peterbhase/ExplanationRoles
Explanations are central to everyday life, and are a topic of growing interest in the AI community. To investigate the process of providing natural language explanations, we leverage the dynamics of the /r/ChangeMyView subreddit to build a dataset with 36K naturally occurring explanations of why an argument is persuasive. We propose a novel word-level prediction task to investigate how explanations selectively reuse, or echo, information from what is being explained (henceforth, explanandum). We develop features to capture the properties of a word in the explanandum, and show that our proposed…
Understanding causality has vital importance for various Natural Language Processing (NLP) applications. Beyond the labeled instances, conceptual explanations of the causality can provide deep understanding of the causal facts to facilitate the causal reasoning process. However, such explanation information still remains absent in existing causal reasoning resources. In this paper, we fill this gap by presenting a human-annotated explainable CAusal REasoning dataset (e-CARE), which contains over 21K causal reasoning questions, together with natural language formed explanations of the causal qu…
TxSum: User-Centered Ethereum Transaction Understanding with Micro-Level Semantic Grounding
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Wuhan University（武汉大学）；University of North Texas（北卡罗来纳州立大学）；Fudan University（复旦大学）
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Microsoft Research（微软研究院）
Interpretability methods are developed to understand the working mechanisms of black-box models, which is crucial to their responsible deployment. Fulfilling this goal requires both that the explanations generated by these methods are correct and that people can easily and reliably understand them. While the former has been addressed in prior work, the latter is often overlooked, resulting in informal model understanding derived from a handful of local explanations. In this paper, we introduce explanation summary (ExSum), a mathematical framework for quantifying model understanding, and propos…
Beihang University（北京航空航天大学）；Tsinghua University（清华大学）；South China University of Technology（华南理工大学）；Sun Yat-sen University（中山大学）；Microsoft Research（微软研究院）；Mila-Québec AI Institute（Mila-魁北克人工智能研究所）；HEC Montréal（蒙特利尔高等商学院）
Recently generating natural language explanations has shown very promising results in not only offering interpretable explanations but also providing additional information and supervision for prediction. However, existing approaches usually require a large set of human annotated explanations for training while collecting a large set of explanations is not only time consuming but also expensive. In this paper, we develop a general framework for interpretable natural language understanding that requires only a small set of human annotated explanations for training. Our framework treats natural …
We describe the task of Visual Understanding and Narration, in which a robot (or agent) generates text for the images that it collects when navigating its environment, by answering open-ended questions, such as 'what happens, or might have happened, here?'
Department of Linguistics, Stanford University（斯坦福大学语言学系）；Google Research（谷歌研究院）；Department of Computer Science, The University of Texas at Austin（德克萨斯大学奥斯汀分校计算机科学系）
A question answering system that in addition to providing an answer provides an explanation of the reasoning that leads to that answer has potential advantages in terms of debuggability, extensibility and trust. To this end, we propose QED, a linguistically informed, extensible framework for explanations in question answering. A QED explanation specifies the relationship between a question and answer according to formal semantic notions such as referential equality, sentencehood, and entailment. We describe and publicly release an expert-annotated dataset of QED explanations built upon a subse…
Columbia University（哥伦比亚大学）；Educational Testing Service（教育考试服务中心）
Figurative language understanding has been recently framed as a recognizing textual entailment (RTE) task (a.k.a. natural language inference, or NLI). However, similar to classical RTE/NLI datasets, the current benchmarks suffer from spurious correlations and annotation artifacts. To tackle this problem, work on NLI has built explanation-based datasets such as e-SNLI, allowing us to probe whether language models are right for the right reasons.Yet no such data exists for figurative language, making it harder to assess genuine understanding of such expressions. To address this issue, we release…
Investigating Co-Constructive Behavior of Large Language Models in Explanation Dialogues
Leibniz University Hannover, Institute of Artificial Intelligence（汉诺威莱布尼茨大学人工智能研究所）；LMU Munich, MCML（慕尼黑大学MCML）；Paderborn University（帕德博恩大学）；Bielefeld University, CITEC（比勒菲尔德大学CITEC）
Pennsylvania State University（宾夕法尼亚州立大学）；University of Washington（华盛顿大学）
Existing self-explaining models typically favor extracting the shortest possible rationales - snippets of an input text "responsible for" corresponding output - to explain the model prediction, with the assumption that shorter rationales are more intuitive to humans. However, this assumption has yet to be validated. Is the shortest rationale indeed the most human-understandable? To answer this question, we design a self-explaining model, LimitedInk, which allows users to extract rationales at any target length. Compared to existing baselines, LimitedInk achieves compatible end-task performance…
Exploring Iterative Enhancement for Improving Learnersourced Multiple-Choice Question Explanations with Large Language Models
Xtracta（埃克斯特拉克塔公司）；School of Computer Science, University of Auckland（奥克兰大学计算机科学学院）；School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；School of Life and Environmental Sciences, University of Sydney（悉尼大学生命与环境科学学院）
LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference
MaiNLP, Center for Information and Language Processing, LMU Munich（慕尼黑大学信息与语言处理中心 MaiNLP）；Munich Center for Machine Learning（慕尼黑机器学习中心）；FNRS, CENTAL, UCLouvain（比利时法语区科学研究基金、鲁汶大学自然语言处理中心）；Faculty of Computer Science and UniVie Doctoral School Computer Science, University of Vienna（维也纳大学计算机科学学院与维也纳大学计算机科学博士学院）
