Exploration and credit assignment under sparse rewards are still challenging problems. We argue that these challenges arise in part due to the intrinsic rigidity of operating at the level of actions. Actions can precisely define how to perform an activity but are ill-suited to describe what activity to perform. Instead, causal effects are inherently composable and temporally abstract, making them ideal for descriptive tasks. By leveraging a hierarchy of causal effects, this study aims to expedite the learning of task-specific behavior and aid exploration. Borrowing counterfactual and normality measures from causal literature, we disentangle controllable effects from effects caused by other dynamics of the environment. We propose CEHRL, a hierarchical method that models the distribution of controllable effects using a Variational Autoencoder. This distribution is used by a high-level policy to 1) explore the environment via random effect exploration so that novel effects are continuously discovered and learned, and to 2) learn task-specific behavior by prioritizing the effects that maximize a given reward function. In comparison to exploring with random actions, experimental results show that random effect exploration is a more efficient mechanism and that by assigning credit to few effects rather than many actions, CEHRL learns tasks more rapidly.
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Cambricon Technologies（寒武纪科技）；Southern University of Science and Technology（南方科技大学）；Institute of Software, CAS（中国科学院软件研究所）；University of Science and Technology of China（中国科学技术大学）
Hierarchical reinforcement learning (HRL) effectively improves agents' exploration efficiency on tasks with sparse reward, with the guide of high-quality hierarchical structures (e.g., subgoals or options). However, how to automatically discover high-quality hierarchical structures is still a great challenge. Previous HRL methods can hardly discover the hierarchical structures in complex environments due to the low exploration efficiency by exploiting the randomness-driven exploration paradigm. To address this issue, we propose CDHRL, a causality-driven hierarchical reinforcement learning fram…
Causal bandit is a nascent learning model where an agent sequentially experiments in a causal network of variables, in order to identify the reward-maximizing intervention. Despite the model's wide applicability, existing analytical results are largely restricted to a parallel bandit version where all variables are mutually independent. We introduce in this work the hierarchical causal bandit model as a viable path towards understanding general causal bandits with dependent variables. The core idea is to incorporate a contextual variable that captures the interaction among all variables with d…
D3HRL: A Distributed Hierarchical Reinforcement Learning Approach Based on Causal Discovery and Spurious Correlation Detection
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机学院）；Intelligent Game and Decision Lab (IGDL)（智能游戏与决策实验室）；College of Artificial Intelligence, Hebei University of Technology（河北工业大学人工智能学院）；School of Electronic information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；Intelligent Game and Decision Lab (IGDL)（智能游戏与决策实验室）；Institute of Military Transportation（军事交通运输研究院）；School of Artificial Intelligence, Hebei University of Technology（河北工业大学人工智能学院）
Hierarchical Reinforcement Learning with Targeted Causal Interventions
School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland（计算机与通信科学系，瑞士洛桑联邦理工学院）；College of Management of Technology, EPFL, Lausanne, Switzerland（技术管理学院，瑞士洛桑联邦理工学院）；Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands（莱顿高级计算机科学研究所（LIACS），莱顿大学）
The recent adaptation of deep neural network-based methods to reinforcement learning and planning domains has yielded remarkable progress on individual tasks. Nonetheless, progress on task-to-task transfer remains limited. In pursuit of efficient and robust generalization, we introduce the Schema Network, an object-oriented generative physics simulator capable of disentangling multiple causes of events and reasoning backward through causes to achieve goals. The richly structured architecture of the Schema Network can learn the dynamics of an environment directly from data. We compare Schema Ne…
In many real-world scenarios, such as gene knockout experiments, targeted interventions are often accompanied by unknown interventions at off-target sites. Moreover, different units can get randomly exposed to different unknown interventions, thereby creating a mixture of interventions. Identifying different components of this mixture can be very valuable in some applications. Motivated by such situations, in this work, we study the problem of identifying all components present in a mixture of interventions on a given causal Bayesian Network. We construct an example to show that, in general, t…
Rethinking State Disentanglement in Causal Reinforcement Learning
Australia Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Shanghai Jiao Tong University（上海交通大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；The University of Melbourne（墨尔本大学）
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
The problem of fair classification can be mollified if we develop a method to remove the embedded sensitive information from the classification features. This line of separating the sensitive information is developed through the causal inference, and the causal inference enables the counterfactual generations to contrast the what-if case of the opposite sensitive attribute. Along with this separation with the causality, a frequent assumption in the deep latent causal model defines a single latent variable to absorb the entire exogenous uncertainty of the causal graph. However, we claim that su…
Causal Foundation Models: Disentangling Physics from Instrument Properties
Massachusetts Institute of Technology（麻省理工学院）；Harvard University（哈佛大学）；New York University（纽约大学）；Flatiron Institute（Flatiron研究所）；The NSF AI Institute for Artificial Intelligence and Fundamental Interactions（NSF人工智能与基本相互作用研究所）
NEC Laboratories America Inc.（NEC美国实验室）；University of Central Florida（中佛罗里达大学）；Amazon Inc.（亚马逊公司）；University of Utah（犹他大学）
In this paper, we propose REASON, a novel framework that enables the automatic discovery of both intra-level (i.e., within-network) and inter-level (i.e., across-network) causal relationships for root cause localization. REASON consists of Topological Causal Discovery and Individual Causal Discovery. The Topological Causal Discovery component aims to model the fault propagation in order to trace back to the root causes. To achieve this, we propose novel hierarchical graph neural networks to construct interdependent causal networks by modeling both intra-level and inter-level non-linear causal …
Carnegie Mellon University（卡内基梅隆大学）；Beijing Technology and Business University（北京工商大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Most causal discovery procedures assume that there are no latent confounders in the system, which is often violated in real-world problems. In this paper, we consider a challenging scenario for causal structure identification, where some variables are latent and they form a hierarchical graph structure to generate the measured variables; the children of latent variables may still be latent and only leaf nodes are measured, and moreover, there can be multiple paths between every pair of variables (i.e., it is beyond tree structure). We propose an estimation procedure that can efficiently locate…
MIMM-X: Disentangling Spurious Correlations for Medical Image Analysis
Medical Image and Data Analysis, University Hospital of Tübingen, Germany（医学图像与数据分析，图宾根大学医院，德国）；Institute for Signal Processing and System Theory, University of Stuttgart, Germany（信号处理与系统理论研究所，斯图加特大学，德国）；Stanford University, Department of Radiology, Stanford, USA（斯坦福大学放射科，斯坦福，美国）
Hierarchical Causal Abduction: A Foundation Framework for Explainable Model Predictive Control
System Dynamics, Chemnitz University of Technology, Chemnitz, Germany（化学特伦茨大学系统动力学系）；Department of Bioresources, Fraunhofer Institute for Molecular Biology and Applied Ecology（弗劳恩霍夫分子生物学与应用生态学研究所生物资源系）
Neurocircuitry-Inspired Hierarchical Graph Causal Attention Networks for Explainable Depression Identification
MOE Frontier Science Center for Brain Science and Brain-machine Integration（脑科学与脑机融合前沿科学中心）；Nanhu Brain-computer Interface Institute（南湖脑机接口研究院）；School of Computer Science and Technology（计算机科学与技术学院）；Qiushi Academy for Advanced Studies（启硕高级研究院）；State Key Laboratory of Brain-machine Intelligence（脑机智能国家重点实验室）；The Department of Neurosurgery, Second Affiliated Hospital, School of Medicine, Zhejiang University Hangzhou, China（浙江大学医学院附属第二医院神经外科）
Identification of Nonlinear Latent Hierarchical Models
Carnegie Mellon University（卡内基梅隆大学）；University of California San Diego（加州大学圣迭戈分校）；Beijing Technology and Business University（北京工商大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Causal Ensemble Agent: Hierarchical Causal Discovery with LLM-guided Expert Reweighting
The University of Melbourne（墨尔本大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Peking University（北京大学）；Adelaide University（阿德莱德大学）；Hong Kong Baptist University（香港浸会大学）；The University of Sydney（悉尼大学）；Carnegie Mellon University（卡内基梅隆大学）
Causal discovery from observational data is an important tool in many branches of science. Under certain assumptions it allows scientists to explain phenomena, predict, and make decisions. In the large sample limit, sound and complete causal discovery algorithms have been previously introduced, where a directed acyclic graph (DAG), or its equivalence class, representing causal relations is searched. However, in real-world cases, only finite training data is available, which limits the power of statistical tests used by these algorithms, leading to errors in the inferred causal model. This is c…
University of Trento（特伦托大学）；Institute for Globally Distributed Open Research and Education (IGDORE)（全球分布式开放研究与教育研究所（IGDORE））；Dublin City University (DCU)（都柏林城市大学（DCU））
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Tensor Vision Technologies（张量视觉技术公司）
Visual objects are composed of a recursive hierarchy of perceptual wholes and parts, whose properties, such as shape, reflectance, and color, constitute a hierarchy of intrinsic causal factors of object appearance. However, object appearance is the compositional consequence of both an object's intrinsic and extrinsic causal factors, where the extrinsic causal factors are related to illumination, and imaging conditions. Therefore, this paper proposes a unified tensor model of wholes and parts, and introduces a compositional hierarchical tensor factorization that disentangles the hierarchical ca…
CausalRAG2: Hierarchical Causal Knowledge Graph Design for RAG
Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, OH, USA（计算机与数据科学系，凯斯西储大学，克利夫兰，OH，USA）；Design and Innovation Department, Case Western Reserve University, Cleveland, OH, USA（设计与创新部门，凯斯西储大学，克利夫兰，OH，USA）
Columbia University（哥伦比亚大学）；Bloomberg L.P.（彭博有限合伙企业）；MILA（蒙特利尔学习算法研究所）；Université de Montréal（蒙特利尔大学）
One of the central elements of any causal inference is an object called structural causal model (SCM), which represents a collection of mechanisms and exogenous sources of random variation of the system under investigation (Pearl, 2000). An important property of many kinds of neural networks is universal approximability: the ability to approximate any function to arbitrary precision. Given this property, one may be tempted to surmise that a collection of neural nets is capable of learning any SCM by training on data generated by that SCM. In this paper, we show this is not the case by disentan…
Learning Discrete Concepts in Latent Hierarchical Models
Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of California San Diego（加州大学圣地亚哥分校）
Disentangled Double Machine Learning for Accurate Causal Effect Estimation
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）；School of Computer and Information Technology, Shanxi University（山西大学计算机与信息学院）
From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；State Key Laboratory of Complex System Modeling and Simulation Technology（复杂系统建模与仿真技术国家重点实验室）；Science & Technology on Integrated Information System Laboratory（信息系统集成技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Wuhan University of Technology（武汉理工大学）
Department of Technology, Management and Economics, Danish Technical University（丹麦技术大学技术、管理与经济系）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Enhancing the Performance of Neural Networks Through Causal Discovery and Integration of Domain Knowledge
The Hong Kong Polytechnic University（香港理工大学）；Sichuan University（四川大学）；The Chinese University of Hong Kong（香港中文大学）；Hong Kong Baptist University（香港浸会大学）；NEOMA Business School（诺欧商学院）
