Learning meaningful causal representations from observations has emerged as a crucial task for facilitating machine learning applications and driving scientific discoveries in fields such as climate science, biology, and physics. This process involves disentangling high-level latent variables and their causal relationships from low-level observations. Previous work in this area that achieves identifiability typically focuses on cases where the observations are either i.i.d. or follow a latent discrete-time process. Nevertheless, many real-world settings require identifying latent variables that are continuous-time stochastic processes (e.g., multivariate point processes). To this end, we develop identifiable causal representation learning for continuous-time latent stochastic point processes. We study its identifiability by analyzing the geometry of the parameter space. Furthermore, we develop MUTATE, an identifiable variational autoencoder framework with a time-adaptive transition module to infer stochastic dynamics. Across simulated and empirical studies, we find that MUTATE can effectively answer scientific questions, such as the accumulation of mutations in genomics and the mechanisms driving neuron spike triggers in response to time-varying dynamics.
Language Agents Meet Causality -- Bridging LLMs and Causal World Models
University of Amsterdam（阿姆斯特丹大学）；Institute for Logic, Language and Computation (ILLC)（逻辑、语言与计算研究所）；University of Edinburgh（爱丁堡大学）；Institute for Language, Cognition and Computation (ILCC)（语言、认知与计算研究所）
Visual representation learning is ubiquitous in various real-world applications, including visual comprehension, video understanding, multi-modal analysis, human-computer interaction, and urban computing. Due to the emergence of huge amounts of multi-modal heterogeneous spatial/temporal/spatial-temporal data in big data era, the lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. The majority of the existing methods tend to fit the original data/variable distributions and ignore the essential causal relations b…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Max Planck ETH Center for Learning Systems（马克斯·普朗克-苏黎世联邦理工学院学习系统中心）；ETH Zurich（苏黎世联邦理工学院）
High-contrast imaging of exoplanets hinges on powerful post-processing methods to denoise the data and separate the signal of a companion from its host star, which is typically orders of magnitude brighter. Existing post-processing algorithms do not use all prior domain knowledge that is available about the problem. We propose a new method that builds on our understanding of the systematic noise and the causal structure of the data-generating process. Our algorithm is based on a modified version of half-sibling regression (HSR), a flexible denoising framework that combines ideas from the field…
Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）；Graduate School of Arts and Sciences, The University of Tokyo（东京大学文理研究生院）
University of Melbourne（墨尔本大学）；University of Tartu（塔尔图大学）
This paper proposes an approach to analyze an event log of a business process in order to generate case-level recommendations of treatments that maximize the probability of a given outcome. Users classify the attributes in the event log into controllable and non-controllable, where the former correspond to attributes that can be altered during an execution of the process (the possible treatments). We use an action rule mining technique to identify treatments that co-occur with the outcome under some conditions. Since action rules are generated based on correlation rather than causation, we the…
Causality Meets Locality: Provably Generalizable and Scalable Policy Learning for Networked Systems
King’s College London（伦敦国王学院）；Eindhoven University of Technology（埃因霍温理工大学）；University of California, San Diego（加州大学圣地亚哥分校）；The Alan Turing Institute（艾伦·图灵研究所）
Drug repurposing is an effective strategy to identify new uses for existing drugs, providing the quickest possible transition from bench to bedside. Existing methods for drug repurposing that mainly focus on pre-clinical information may exist translational issues when applied to human beings. Real world data (RWD), such as electronic health records and insurance claims, provide information on large cohorts of users for many drugs. Here we present an efficient and easily-customized framework for generating and testing multiple candidates for drug repurposing using a retrospective analysis of RW…
Georgia Institute of Technology（佐治亚理工学院）；IBM Research（IBM研究院）
Discovering and exploiting the causality in deep neural networks (DNNs) are crucial challenges for understanding and reasoning causal effects (CE) on an explainable visual model. "Intervention" has been widely used for recognizing a causal relation ontologically. In this paper, we propose a causal inference framework for visual reasoning via do-calculus. To study the intervention effects on pixel-level features for causal reasoning, we introduce pixel-wise masking and adversarial perturbation. In our framework, CE is calculated using features in a latent space and perturbed prediction from a D…
National University of Singapore（新加坡国立大学）；London School of Economics and Political Science（伦敦政治经济学院）
This work extends causal inference with stochastic confounders. We propose a new approach to variational estimation for causal inference based on a representer theorem with a random input space. We estimate causal effects involving latent confounders that may be interdependent and time-varying from sequential, repeated measurements in an observational study. Our approach extends current work that assumes independent, non-temporal latent confounders, with potentially biased estimators. We introduce a simple yet elegant algorithm without parametric specification on model components. Our method a…
We provide some basic and sensible definitions of different types of digital twins and recommendations on when and how to use them. Following up on our recent publication of the Learning Causal Digital Twin, this article reports on a stochastic formulation and solution of the problem. Structural Vector Autoregressive Model (SVAR) for Causal estimation is recast as a state-space model. Kalman filter (and smoother) is then employed to estimate causal factors in a system of connected machine bearings. The previous neural network algorithm and Kalman Smoother produced very similar results; however…
Causal inference methods are widely applied in various decision-making domains such as precision medicine, optimal policy and economics. Central to these applications is the treatment effect estimation of intervention strategies. Current estimation methods are mostly restricted to the deterministic treatment, which however, is unable to address the stochastic space treatment policies. Moreover, previous methods can only make binary yes-or-no decisions based on the treatment effect, lacking the capability of providing fine-grained effect estimation degree to explain the process of decision maki…
National Institute of Advanced Studies（国家高级研究院）；Indian Institute of Science（印度科学学院）
Chaos and Noise are ubiquitous in the Brain. Inspired by the chaotic firing of neurons and the constructive role of noise in neuronal models, we for the first time connect chaos, noise and learning. In this paper, we demonstrate Stochastic Resonance (SR) phenomenon in Neurochaos Learning (NL). SR manifests at the level of a single neuron of NL and enables efficient subthreshold signal detection. Furthermore, SR is shown to occur in single and multiple neuronal NL architecture for classification tasks - both on simulated and real-world spoken digit datasets. Intermediate levels of noise in neur…
Stochastic gradient descent (SGD) provides a simple and efficient way to solve a broad range of machine learning problems. Here, we focus on distribution regression (DR), involving two stages of sampling: Firstly, we regress from probability measures to real-valued responses. Secondly, we sample bags from these distributions for utilizing them to solve the overall regression problem. Recently, DR has been tackled by applying kernel ridge regression and the learning properties of this approach are well understood. However, nothing is known about the learning properties of SGD for two stage samp…
Local causal states are latent representations that capture organized pattern and structure in complex spatiotemporal systems. We expand their functionality, framing them as spacetime autoencoders. Previously, they were only considered as maps from observable spacetime fields to latent local causal state fields. Here, we show that there is a stochastic decoding that maps back from the latent fields to observable fields. Furthermore, their Markovian properties define a stochastic dynamic in the latent space. Combined with stochastic decoding, this gives a new method for forecasting spacetime fi…
Causal inference methods are widely applied in various decision-making domains such as precision medicine, optimal policy and economics. Central to causal inference is the treatment effect estimation of intervention strategies, such as changes in drug dosing and increases in financial aid. Existing methods are mostly restricted to the deterministic treatment and compare outcomes under different treatments. However, they are unable to address the substantial recent interest of treatment effect estimation under stochastic treatment, e.g., "how all units health status change if they adopt 50\% do…
Stochastic Causal Programming for Bounding Treatment Effects
Helmholtz AI（亥姆霍兹人工智能研究所）；Helmholtz Munich（亥姆霍兹慕尼黑中心）；Technical University Munich（慕尼黑工业大学）；University College London（伦敦大学学院）；King’s College London（伦敦国王学院）
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Saarland University（萨尔大学）
The algorithmic Markov condition states that the most likely causal direction between two random variables X and Y can be identified as that direction with the lowest Kolmogorov complexity. Due to the halting problem, however, this notion is not computable. We hence propose to do causal inference by stochastic complexity. That is, we propose to approximate Kolmogorov complexity via the Minimum Description Length (MDL) principle, using a score that is mini-max optimal with regard to the model class under consideration. This means that even in an adversarial setting, such as when the true distri…
Causal Discovery from Heteroscedastic Stochastic Dynamical Systems under Imperfect Physical Models
Department of Mechanical & Industrial Engineering（机械与工业工程系）；Northeastern University（东北大学）；Department of Industrial Engineering and Management Sciences（工业工程与管理科学系）；Department of Mechanical Engineering（机械工程系）；Northwestern University（西北大学）
Signature Kernel Conditional Independence Tests in Causal Discovery for Stochastic Processes
Technical University of Munich（慕尼黑工业大学）；Helmholtz Munich（慕尼黑亥姆霍兹中心）；Munich Center for Machine Learning（慕尼黑机器学习中心）；University of Oxford（牛津大学）；Lund University（隆德大学）；Imperial College London（伦敦帝国学院）
Stochastic blockmodels (SBM) and their variants, $e.g.$, mixed-membership and overlapping stochastic blockmodels, are latent variable based generative models for graphs. They have proven to be successful for various tasks, such as discovering the community structure and link prediction on graph-structured data. Recently, graph neural networks, $e.g.$, graph convolutional networks, have also emerged as a promising approach to learn powerful representations (embeddings) for the nodes in the graph, by exploiting graph properties such as locality and invariance. In this work, we unify these two di…
Future wireless services, such as the metaverse require high information rate, reliability, and low latency. Multi-user wireless systems can meet such requirements by utilizing the abundant terahertz bandwidth with a massive number of antennas, creating narrow beamforming solutions. However, existing solutions lack proper modeling of channel dynamics, resulting in inaccurate beamforming solutions in high-mobility scenarios. Herein, a dynamic, semantically aware beamforming solution is proposed for the first time, utilizing novel artificial intelligence algorithms in variational causal inferenc…
Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学研究所）；Korteweg-de Vries Institute, University of Amsterdam（阿姆斯特丹大学科尔特韦格-德弗里斯研究所）；Pattern Recognition Laboratory, Delft University of Technology（代尔夫特理工大学模式识别实验室）
Dynamical systems are widely used in science and engineering to model systems consisting of several interacting components. Often, they can be given a causal interpretation in the sense that they not only model the evolution of the states of the system's components over time, but also describe how their evolution is affected by external interventions on the system that perturb the dynamics. We introduce the formal framework of structural dynamical causal models (SDCMs) that explicates the causal semantics of the system's components as part of the model. SDCMs represent a dynamical system as a …
A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization
University of Klagenfurt（克拉根福大学）；Silicon Austria Labs GmbH（Silicon Austria Labs 有限公司）
This article proposes and evaluates a technique to predict the level of interference in wireless networks. We design a recursive predictor that estimates future interference values by filtering measured interference at a given location. The predictor's parameterization is done offline by translating the autocorrelation of interference into an autoregressive moving average (ARMA) representation. This ARMA model is inserted into a steady-state Kalman filter enabling nodes to predict with low computational effort. Results show a good accuracy of predicted values versus true values for relevant ti…
LG AI Research（LG人工智能研究院）；KAIST（韩国科学技术院）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）
To overcome the quadratic cost of self-attention, recent works have proposed various sparse attention modules, most of which fall under one of two groups: 1) sparse attention under a hand-crafted patterns and 2) full attention followed by a sparse variant of softmax such as $α$-entmax. Unfortunately, the first group lacks adaptability to data while the second still requires quadratic cost in training. In this work, we propose SBM-Transformer, a model that resolves both problems by endowing each attention head with a mixed-membership Stochastic Block Model (SBM). Then, each attention head data-…
In wireless networks, many problems can be formulated as subset selection problems where the goal is to select a subset from the ground set with the objective of maximizing some objective function. These problems are typically NP-hard and hence solved through carefully constructed heuristics, which are themselves mostly NP-complete and thus not easily applicable to large networks. On the other hand, subset selection problems occur in slightly different context in machine learning (ML) where the goal is to select a subset of high quality yet diverse items from a ground set. In this paper, we in…
Ant Group（蚂蚁集团）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）
We consider a sequential decision making problem where the agent faces the environment characterized by the stochastic discrete events and seeks an optimal intervention policy such that its long-term reward is maximized. This problem exists ubiquitously in social media, finance and health informatics but is rarely investigated by the conventional research in reinforcement learning. To this end, we present a novel framework of the model-based reinforcement learning where the agent's actions and observations are asynchronous stochastic discrete events occurring in continuous-time. We model the d…
