Theory-of-Mind (ToM) ability possessed by Large Language Models (LLMs) has been shown to be limited. Most existing methods for improving ToM in LLMs adopt zero-shot prompting, and they face challenges including poor performance in complex ToM reasoning tasks and an inability to handle non-narrative contexts. We propose a zero-shot prompting method named Constrained Chain-of-ToM (CCoToM) that leverages domain knowledge and the causal relations between ToM dimensions to address these limitations. Specifically, CCoToM guides LLMs to construct explicit reasoning chains by first prompting LLMs to infer related ToM dimensions (e.g., belief). Afterward, CCoToM prompts LLMs to infer the queried ToM dimension based on the generated related ToM dimensions and corresponding causal relations. Additionally, CCoToM adaptively imposes constraints on prompts to introduce inductive biases and improve consistency between ToM dimensions. Besides narratives, CCoToM can also handle non-narrative contexts like conversations. Extensive experiments show that CCoToM consistently outperforms previous state-of-the-art methods by large margins across all LLMs and datasets used. We also conduct in-depth analyses to gain deeper insights into CCoToM. We have made our code publicly available.
PHAnToM: Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Models
Institute of Data Science(数据科学研究所);NUS Centre for Trusted Internet and Community, National University of Singapore(新加坡国立大学可信互联网与社区中心);Amazon(亚马逊);Stanford University(斯坦福大学);Tsinghua University(清华大学)
MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人机混合增强智能国家重点实验室、视觉信息与应用国家工程研究中心、人工智能与机器人研究所);Nankai University(南开大学);The Hong Kong University of Science and Technology(Guangzhou)(香港科技大学(广州));School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件学院)
Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLM
University of Southern California(南加州大学);The University of Queensland(昆士兰大学);University of California, San Diego(加州大学圣地亚哥分校);University of California, Merced(加州大学默塞德分校)
Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
Bar-Ilan University(巴伊兰大学);University of British Columbia(不列颠哥伦比亚大学);Vector Institute for AI(Vector人工智能研究所);Carnegie Mellon University(卡内基梅隆大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of Washington(华盛顿大学)
Mind the Gap: Benchmarking LLM Uncertainty and Calibration with Specialty-Aware Clinical QA and Reasoning-Based Behavioural Features
Department of Medical Informatics, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, The Netherlands(医学信息学系,阿姆斯特丹大学医学中心,阿姆斯特丹大学,阿姆斯特丹,荷兰);Amsterdam Public Health, Methodology, Amsterdam, The Netherlands(阿姆斯特丹公共健康,方法学,阿姆斯特丹,荷兰)
EvolvTrip: Enhancing Literary Character Understanding with Temporal Theory-of-Mind Graphs
The University of Manchester(曼彻斯特大学);King’s College London(伦敦国王学院);Huawei London Research Centre(华为伦敦研究中心);Huawei Technologies Co., Ltd.(华为技术有限公司);The Alan Turing Institute(艾伦·图灵研究所)
Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
Hebrew University of Jerusalem(海法大学);MPI for Biological Cybernetics(生物感知研究所);Centre for AI and Machine Learning, AU(人工智能与机器学习中心);IoPPN, King’s College London(国王学院伦敦人工智能与感知神经科学中心)
Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning
Saarland University(萨尔兰大学);Deutsches Forschungszentrum für Künstliche Intelligenz(德国人工智能研究中心);Télécom Paris(巴黎电信学院);Max Planck Institute for Informatics(马克斯·普朗克 informatics 研究所)
Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Bonn-Aachen International Center for Information Technology (b-it)(波恩-埃森国际信息科技中心);University of Bonn(波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所);Research Center Trustworthy Data Science and Security (RC-Trust)(可信数据科学与安全研究中心);University of Duisburg-Essen(埃森-杜伊斯堡大学)
本研究通过扰动任务和链式推理提示探讨大语言模型的理论 of mind能力,发现其鲁棒性受扰动影响显著,CoT提示虽提升整体表现,但对某些扰动类别却降低准确性。