Controllable text generation concerns two fundamental tasks of wide applications, namely generating text of given attributes (i.e., attribute-conditional generation), and minimally editing existing text to possess desired attributes (i.e., text attribute transfer). Extensive prior work has largely studied the two problems separately, and developed different conditional models which, however, are prone to producing biased text (e.g., various gender stereotypes). This paper proposes to formulate controllable text generation from a principled causal perspective which models the two tasks with a unified framework. A direct advantage of the causal formulation is the use of rich causality tools to mitigate generation biases and improve control. We treat the two tasks as interventional and counterfactual causal inference based on a structural causal model, respectively. We then apply the framework to the challenging practical setting where confounding factors (that induce spurious correlations) are observable only on a small fraction of data. Experiments show significant superiority of the causal approach over previous conditional models for improved control accuracy and reduced bias.
Harbin Institute of Technology（哈尔滨工业大学）；Peng Cheng Laboratory（鹏城实验室）
Multi-aspect controllable text generation is a more challenging and practical task than single-aspect control. Existing methods achieve complex multi-aspect control by fusing multiple controllers learned from single-aspect, but suffer from attribute degeneration caused by the mutual interference of these controllers. To address this, we provide observations on attribute fusion from a distributional perspective and propose to directly search for the intersection areas of multiple attribute distributions as their combination for generation. Our method first estimates the attribute space with an …
The Hong Kong University of Science and Technology（香港科技大学）；Zhejiang Laboratory（之江实验室）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
This paper proposes a Disentangled gEnerative cAusal Representation (DEAR) learning method under appropriate supervised information. Unlike existing disentanglement methods that enforce independence of the latent variables, we consider the general case where the underlying factors of interests can be causally related. We show that previous methods with independent priors fail to disentangle causally related factors even under supervision. Motivated by this finding, we propose a new disentangled learning method called DEAR that enables causal controllable generation and causal representation le…
University of Melbourne（墨尔本大学）；University of Tartu（塔尔图大学）
This paper proposes an approach to analyze an event log of a business process in order to generate case-level recommendations of treatments that maximize the probability of a given outcome. Users classify the attributes in the event log into controllable and non-controllable, where the former correspond to attributes that can be altered during an execution of the process (the possible treatments). We use an action rule mining technique to identify treatments that co-occur with the outcome under some conditions. Since action rules are generated based on correlation rather than causation, we the…
With the increasing adoption of predictive models trained using machine learning across a wide range of high-stakes applications, e.g., health care, security, criminal justice, finance, and education, there is a growing need for effective techniques for explaining such models and their predictions. We aim to address this problem in settings where the predictive model is a black box; That is, we can only observe the response of the model to various inputs, but have no knowledge about the internal structure of the predictive model, its parameters, the objective function, and the algorithm used t…
Pennsylvania State University（宾夕法尼亚州立大学）；Purdue University（普渡大学）
As virtually all aspects of our lives are increasingly impacted by algorithmic decision making systems, it is incumbent upon us as a society to ensure such systems do not become instruments of unfair discrimination on the basis of gender, race, ethnicity, religion, etc. We consider the problem of determining whether the decisions made by such systems are discriminatory, through the lens of causal models. We introduce two definitions of group fairness grounded in causality: fair on average causal effect (FACE), and fair on average causal effect on the treated (FACT). We use the Rubin-Neyman pot…
The term `spurious correlations' has been used in NLP to informally denote any undesirable feature-label correlations. However, a correlation can be undesirable because (i) the feature is irrelevant to the label (e.g. punctuation in a review), or (ii) the feature's effect on the label depends on the context (e.g. negation words in a review), which is ubiquitous in language tasks. In case (i), we want the model to be invariant to the feature, which is neither necessary nor sufficient for prediction. But in case (ii), even an ideal model (e.g. humans) must rely on the feature, since it is necess…
Identifiable Latent Polynomial Causal Models Through the Lens of Change
Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；School of Computer Science and Engineering, The University of New South Wales（新南威尔士大学计算机科学与工程学院）；School of Mathematics and Statistics, The University of Melbourne（墨尔本大学数学与统计学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Halicioğlu Data Science Institute (HDSI), University of California San Diego（加州大学圣地亚哥分校哈利西奥格鲁数据科学研究所）；Carnegie Mellon University（卡内基梅隆大学）
Causal Pre-training Under the Fairness Lens: An Empirical Study of TabPFN
University of Bergen Centre for the Science of Learning \& Technology (SLATE) Bergen Norway；Department of Computer Science University of Oxford；Alan Turing Institute Oxford United Kingdom；University of Bergen；University of Oxford；Alan Turing Institute
Wasserstein Distributionally Robust Optimization Through the Lens of Structural Causal Models and Individual Fairness
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（图宾根人工智能中心）；Mila Québec AI Institute（魁北克人工智能研究所）；McGill University（麦吉尔大学）
Complex environments and tasks pose a difficult problem for holistic end-to-end learning approaches. Decomposition of an environment into interacting controllable and non-controllable objects allows supervised learning for non-controllable objects and universal value function approximator learning for controllable objects. Such decomposition should lead to a shorter learning time and better generalisation capability. Here, we consider arcade-game environments as sets of interacting objects (controllable, non-controllable) and propose a set of functional modules that are specialized on masterin…
Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens
University of California, San Francisco（加利福尼亚大学旧金山分校）；U.S. Food and Drug Administration, Center for Devices and Radiological Health（美国食品药品监督管理局医疗器械和放射健康中心）
What is a good matching of probability measures? A counterfactual lens on transport maps
Institut Elie Cartan de Lorraine, Université de Lorraine（伊莱·卡坦研究所（洛林）、洛林大学）；Université Paris-Saclay, CNRS, Inria, Laboratoire de mathématiques d’Orsay（巴黎-萨克雷大学、国家科学研究中心、法国国家信息与自动化技术研究院、奥赛数学实验室）
Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Cambridge（剑桥大学）；ELLIS Institute Tübingen（埃利斯研究所蒂宾根分所）；Tübingen AI Center（蒂宾根人工智能中心）
Causal Mechanism Estimation in Multi-Sensor Systems Across Multiple Domains
Bosch Center for Artificial Intelligence（博世人工智能中心）；Institute of Industrial Manufacturing and Management IFF, University of Stuttgart（工业制造与管理研究所）；Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所）
Data-driven modelling and synthesis of motion is an active research area with applications that include animation, games, and social robotics. This paper introduces a new class of probabilistic, generative, and controllable motion-data models based on normalising flows. Models of this kind can describe highly complex distributions, yet can be trained efficiently using exact maximum likelihood, unlike GANs or VAEs. Our proposed model is autoregressive and uses LSTMs to enable arbitrarily long time-dependencies. Importantly, is is also causal, meaning that each pose in the output sequence is gen…
University of Tübingen（蒂宾根大学）；Technical University of Munich（慕尼黑工业大学）；Munich Data Science Institute（慕尼黑数据科学研究所）
We study the problem of learning causal models from observational data through the lens of interpolation and its counterpart -- regularization. A large volume of recent theoretical, as well as empirical work, suggests that, in highly complex model classes, interpolating estimators can have good statistical generalization properties and can even be optimal for statistical learning. Motivated by an analogy between statistical and causal learning recently highlighted by Janzing (2019), we investigate whether interpolating estimators can also learn good causal models. To this end, we consider a si…
Controllable Text Generation with Residual Memory Transformer
School of Computer Science & Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
Causality in the human niche: lessons for machine learning
Dept. of Computer Science, Rochester Institute of Technology（计算机科学系，罗切斯特理工学院）；Cognitive Science PhD Program, Rochester Institute of Technology（认知科学博士项目，罗切斯特理工学院）；Center for Vision Science, University of Rochester（视觉科学中心，罗切斯特大学）；Dept. of Neurobiology, University of Pennsylvania（神经生物学系，宾夕法尼亚大学）；CIFAR LMB
Causal Structure Representation Learning of Confounders in Latent Space for Recommendation
MIC Lab, College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院MIC实验室）；Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications（北京邮电大学可信分布式计算与服务重点实验室）；Institute of Artificial Intelligence, Beihang University and Zhongguancun Laboratory（北航人工智能研究院和中关村实验室）
From Baseline to Follow-Up: Counterfactual Spine DXA Image Synthesis in UK Biobank Using a Causal Hierarchical Variational Autoencoder
School of Electronics and Computer Science（电子与计算机科学学院）；University of Southampton（萨塞克斯大学）；MRC Lifecourse Epidemiology Centre（英国医学研究理事会生命周期流行病学中心）；University of Southampton, Southampton General Hospital（萨塞克斯大学索马塞特医院）；Computer Science University of Southampton（计算机科学萨塞克斯大学）
Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
Imperial College London（帝国理工学院）；United Kingdom（英国）；Department of Computing（计算机系）
The societal issue of digital hostility has previously attracted a lot of attention. The topic counts an ample body of literature, yet remains prominent and challenging as ever due to its subjective nature. We posit that a better understanding of this problem will require the use of causal inference frameworks. This survey summarises the relevant research that revolves around estimations of causal effects related to online hate speech. Initially, we provide an argumentation as to why re-establishing the exploration of hate speech in causal terms is of the essence. Following that, we give an ov…
University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
Unconscious biases continue to be prevalent in modern text and media, calling for algorithms that can assist writers with bias correction. For example, a female character in a story is often portrayed as passive and powerless ("She daydreams about being a doctor") while a man is portrayed as more proactive and powerful ("He pursues his dream of being a doctor"). We formulate *Controllable Debiasing*, a new revision task that aims to rewrite a given text to correct the implicit and potentially undesirable bias in character portrayals. We then introduce PowerTransformer as an approach that debia…
When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality
University of Potsdam（波茨坦大学）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
