Counterfactual thinking describes a psychological phenomenon that people re-infer the possible results with different solutions about things that have already happened. It helps people to gain more experience from mistakes and thus to perform better in similar future tasks. This paper investigates the counterfactual thinking for agents to find optimal decision-making strategies in multi-agent reinforcement learning environments. In particular, we propose a multi-agent deep reinforcement learning model with a structure which mimics the human-psychological counterfactual thinking process to improve the competitive abilities for agents. To this end, our model generates several possible actions (intent actions) with a parallel policy structure and estimates the rewards and regrets for these intent actions based on its current understanding of the environment. Our model incorporates a scenario-based framework to link the estimated regrets with its inner policies. During the iterations, our model updates the parallel policies and the corresponding scenario-based regrets for agents simultaneously. To verify the effectiveness of our proposed model, we conduct extensive experiments on two different environments with real-world applications. Experimental results show that counterfactual thinking can actually benefit the agents to obtain more accumulative rewards from the environments with fair information by comparing to their opponents while keeping high performing efficiency.
ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of British Columbia（不列颠哥伦比亚大学）；University College London（伦敦大学学院）
There exists an apparent negative correlation between performance and interpretability of deep learning models. In an effort to reduce this negative correlation, we propose a Born Identity Network (BIN), which is a post-hoc approach for producing multi-way counterfactual maps. A counterfactual map transforms an input sample to be conditioned and classified as a target label, which is similar to how humans process knowledge through counterfactual thinking. For example, a counterfactual map can localize hypothetical abnormalities from a normal brain image that may cause it to be diagnosed with a…
Evaluating the Impact of Pulse Oximetry Bias in Machine Learning under Counterfactual Thinking
Faculty of Engineering, University of Porto（波尔图大学工程学院）；Institute for Systems and Computer Engineering, Technology and Science（系统与计算机工程、技术与科学研究所）；Duke University（杜克大学）；Massachusetts Institute of Technology（麻省理工学院）
Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models
Mila – Québec AI Institute（魁北克AI研究所）；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）；LawZero（法零）；LLNL（劳伦斯利弗莫尔国家实验室）；University of Edinburgh（爱丁堡大学）；CIFAR AI Chair（CIFAR人工智能 chair）；CIFAR Fellow
University of Science and Technology of China（中国科学技术大学）；New York University（纽约大学）
Counterfactual Regret Minimization (CFR) has achieved many fascinating results in solving large-scale Imperfect Information Games (IIGs). Neural network approximation CFR (neural CFR) is one of the promising techniques that can reduce computation and memory consumption by generalizing decision information between similar states. Current neural CFR algorithms have to approximate cumulative regrets. However, efficient and accurate approximation in a large-scale IIG is still a tough challenge. In this paper, a new CFR variant, Recursive CFR (ReCFR), is proposed. In ReCFR, Recursive Substitute Val…
