CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models
UCLA（加州大学洛杉矶分校）；University of Michigan（密歇根大学）；Oregon State University（俄勒冈州立大学）；Beijing Institute for General AI (BIGAI)（北京通用人工智能研究院）；Tsinghua University（清华大学）；Peking University（北京大学）
Arjun R. Akula, Keze Wang, Changsong Liu, Sari Saba-Sadiya, Hongjing Lu, Sinisa Todorovic, Joyce Chai, Song-Chun Zhu
备注：
Accepted by iScience Cell Press Journal 2021. arXiv admin note: text overlap with arXiv:1909.06907
We propose CX-ToM, short for counterfactual explanations with theory-of mind, a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN). In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process, i.e. dialog, between the machine and human user. More concretely, our CX-ToM framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user. To do this, we use Theory of Mind (ToM) which helps us in explicitly modeling human's intention, machine's mind as inferred by the human as well as human's mind as inferred by the machine. Moreover, most state-of-the-art XAI frameworks provide attention (or heat map) based explanations. In our work, we show that these attention based explanations are not sufficient for increasing human trust in the underlying CNN model. In CX-ToM, we instead use counterfactual explanations called fault-lines which we define as follows: given an input image I for which a CNN classification model M predicts class c_pred, a fault-line identifies the minimal semantic-level features (e.g., stripes on zebra, pointed ears of dog), referred to as explainable concepts, that need to be added to or deleted from I in order to alter the classification category of I by M to another specified class c_alt. We argue that, due to the iterative, conceptual and counterfactual nature of CX-ToM explanations, our framework is practical and more natural for both expert and non-expert users to understand the internal workings of complex deep learning models. Extensive quantitative and qualitative experiments verify our hypotheses, demonstrating that our CX-ToM significantly outperforms the state-of-the-art explainable AI models.
We present a new explainable AI (XAI) framework aimed at increasing justified human trust and reliance in the AI machine through explanations. We pose explanation as an iterative communication process, i.e. dialog, between the machine and human user. More concretely, the machine generates sequence of explanations in a dialog which takes into account three important aspects at each dialog turn: (a) human's intention (or curiosity); (b) human's understanding of the machine; and (c) machine's understanding of the human user. To do this, we use Theory of Mind (ToM) which helps us in explicitly mod…
Mind the Perspective: Let's Reason Recursively for Theory of Mind
School of Computing and Information Systems, The University of Melbourne, Australia（墨尔本大学计算与信息系统学院）；SensiLab, Monash University, Australia（蒙纳士大学SensiLab）
MMToM-QA: Multimodal Theory of Mind Question Answering
New York University（纽约大学）；Harvard University（哈佛大学）；Massachusetts Institute of Technology（麻省理工学院）；UC San Diego（加州大学圣迭戈分校）；University of Virginia（弗吉尼亚大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
Hebrew University of Jerusalem（海法大学）；MPI for Biological Cybernetics（生物感知研究所）；Centre for AI and Machine Learning, AU（人工智能与机器学习中心）；IoPPN, King’s College London（国王学院伦敦人工智能与感知神经科学中心）
National University of Singapore（新加坡国立大学）；AI Singapore（新加坡人工智能计划）；Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究院）
Recent advances in Artificial Intelligence have produced agents that can beat human world champions at games like Go, Starcraft, and Dota2. However, most of these models do not seem to play in a human-like manner: People infer others' intentions from their behaviour, and use these inferences in scheming and strategizing. Here, using a Bayesian Theory of Mind (ToM) approach, we investigated how much an explicit representation of others' intentions improves performance in a cooperative game. We compared the performance of humans playing with optimal-planning agents with and without ToM, in a coo…
Great advancements have been achieved in the field of robotics, however, main challenges remain, including building robots with an adaptive Theory of Mind (ToM). In the present paper, seven current robotic architectures for human-robot interactions were described as well as four main functional advantages of equipping robots with an adaptive ToM. The aim of the present paper was to determine in which way and how often ToM features are integrated in the architectures analyzed, and if they provide robots with the associated functional advantages. Our assessment shows that different methods are u…
Machine learning of Theory of Mind (ToM) is essential to build social agents that co-live with humans and other agents. This capacity, once acquired, will help machines infer the mental states of others from observed contextual action trajectories, enabling future prediction of goals, intention, actions and successor representations. The underlying mechanism for such a prediction remains unclear, however. Inspired by the observation that humans often infer the character traits of others, then use it to explain behaviour, we propose a new neural ToM architecture that learns to generate a latent…
ToMBench: Benchmarking Theory of Mind in Large Language Models
Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；IIE, CAS（中国科学院自动化研究所）；Tianjin University（天津大学）；Northwest Minzu University（西北民族大学）；Nankai University（南开大学）；Beijing Institute of Technology（北京理工大学）
Social reasoning necessitates the capacity of theory of mind (ToM), the ability to contextualise and attribute mental states to others without having access to their internal cognitive structure. Recent machine learning approaches to ToM have demonstrated that we can train the observer to read the past and present behaviours of other agents and infer their beliefs (including false beliefs about things that no longer exist), goals, intentions and future actions. The challenges arise when the behavioural space is complex, demanding skilful space navigation for rapidly changing contexts for an ex…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Currently, in the study of multiagent systems, the intentions of agents are usually ignored. Nonetheless, as pointed out by Theory of Mind (ToM), people regularly reason about other's mental states, including beliefs, goals, and intentions, to obtain performance advantage in competition, cooperation or coalition. However, due to its intrinsic recursion and intractable modeling of distribution over belief, integrating ToM in multiagent planning and decision making is still a challenge. In this paper, we incorporate ToM in multiagent partially observable Markov decision process (POMDP) and propo…
ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind
The Graduate Center, CUNY（纽约城市大学研究生中心）；Toyota Technological Institute at Chicago（丰田芝加哥技术学院）；Basque Center on Cognition, Brain and Language (BCBL)（巴斯克认知、大脑与语言中心）
Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planning
Department of Psychology, University of Toronto（多伦多大学心理学系）；Vector Institute（向量研究所）；Schwartz Reisman Institute of Technology & Society, University of Toronto（多伦多大学科技与社会研究所）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）
Readable Minds: Emergent Theory-of-Mind-Like Behavior in LLM Poker Agents
Department of Oncology, Koo Foundation Sun Yat-Sen Cancer Center（辜公亮基金会和信治癌中心医院肿瘤科）；Department of Digital Content and Technologies, National Chengchi University（国立政治大学数位内容与科技学系）
Theory of mind (ToM; Premack & Woodruff, 1978) broadly refers to humans' ability to represent the mental states of others, including their desires, beliefs, and intentions. We propose to train a machine to build such models too. We design a Theory of Mind neural network -- a ToMnet -- which uses meta-learning to build models of the agents it encounters, from observations of their behaviour alone. Through this process, it acquires a strong prior model for agents' behaviour, as well as the ability to bootstrap to richer predictions about agents' characteristics and mental states using only a sma…
Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Bonn-Aachen International Center for Information Technology (b-it)（波恩-埃森国际信息科技中心）；University of Bonn（波恩大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）；Research Center Trustworthy Data Science and Security (RC-Trust)（可信数据科学与安全研究中心）；University of Duisburg-Essen（埃森-杜伊斯堡大学）
本研究通过扰动任务和链式推理提示探讨大语言模型的理论 of mind能力，发现其鲁棒性受扰动影响显著，CoT提示虽提升整体表现，但对某些扰动类别却降低准确性。
CoSToM:Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Models
National Engineering Research Center for Big Data Technology and System, Services Computing Technology and System Lab, Cluster and Grid Computing Lab, Huazhong University of Science and Technology（华中科技大学国家大数据技术与系统工程技术研究中心、服务计算技术与系统实验室、集群与网格计算实验室）；Singapore Management University（新加坡管理大学）
