As large language models (LLMs) see greater use in academic and commercial settings, there is increasing interest in methods that allow language models to generate texts aligned with human preferences. In this paper, we present an initial exploration of language model optimization for human preferences from direct outcome datasets, where each sample consists of a text and an associated numerical outcome measuring the reader's response. We first propose that language model optimization should be viewed as a causal problem to ensure that the model correctly learns the relationship between the text and the outcome. We formalize this causal language optimization problem, and we develop a method--causal preference optimization (CPO)--that solves an unbiased surrogate objective for the problem. We further extend CPO with doubly robust CPO (DR-CPO), which reduces the variance of the surrogate objective while retaining provably strong guarantees on bias. Finally, we empirically demonstrate the effectiveness of (DR-)CPO in optimizing state-of-the-art LLMs for human preferences on direct outcome data, and we validate the robustness of DR-CPO under difficult confounding conditions.
Causal Structure Representation Learning of Confounders in Latent Space for Recommendation
MIC Lab, College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院MIC实验室）；Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications（北京邮电大学可信分布式计算与服务重点实验室）；Institute of Artificial Intelligence, Beihang University and Zhongguancun Laboratory（北航人工智能研究院和中关村实验室）
FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
LMU Munich（慕尼黑大学）；Columbia University（哥伦比亚大学）；University of Southern California（南加州大学）；University of Oxford（牛津大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
针对 DPO 难以纠正错误排序偏好对的问题，本文提出 FocalPO，通过 Focal Loss 式调制因子动态降权错误样本、强化正确样本，并在多个基准和模型上优于 DPO 变体。
Optimizing Experimental Design for Causal Effect Estimation with Partial Measurements
Fraunhofer Institute for Cognitive Systems IKS（弗劳恩霍夫认知系统研究所）；TUM School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）
Optimizing Large Language Models for Causality Assessment in Pharmacovigilance: Developing a Performance Metric as Objective for Bayesian Hyperparameter Optimization
Department of Drug Design and Pharmacology, University of Copenhagen（哥本哈根大学药物设计与药理学系）；Safety Operations, Novo Nordisk（诺和制药安全运营部）；Clinical Development Centre Denmark, Novo Nordisk（丹麦临床开发中心，诺和制药）；Statistical Data Science Lead, Pfizer（辉瑞统计数据科学负责人）
PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization
School of Data Science and Engineering, East China Normal University（华东师范大学数据科学与工程学院）；Chinese Academy of Sciences（中国科学院）；Baidu Inc（百度公司）；Peking University（北京大学）
SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization
Shanghai Jiao Tong University（上海交通大学）；NJIT（新泽西理工学院）；Jilin University（吉林大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Meta-Router: Bridging Gold-standard and Preference-based Evaluations in Large Language Model Routing
Department of Statistics, Indiana University Bloomington（印第安纳大学布卢明顿分校统计系）；Department of Statistics, North Carolina State University（北卡罗来纳州立大学统计系）；Department of Biostatistics, The University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心生物统计学系）
Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment
Artificial Intelligence Research Laboratory, Pennsylvania State University（宾夕法尼亚州立大学人工智能研究实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Tencent AI Lab（腾讯人工智能实验室）
Faithful or Fabricated? A Causal Framework for Rationalization Bias in LLM Judges
School of Computing and Electrical Engineering（计算与电子工程学院）；Indian Institute of Technology (IIT) Mandi（印度理工学院（IIT）曼迪）；London U.K.（伦敦英国）；Warwick Manufacturing Group U.K.（沃里克制造集团英国）
Just what the doctor ordered: An evaluation of provider preference-based Instrumental Variable methods in observational studies, with application for comparative effectiveness of type 2 diabetes therapy
Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of California, San Diego（加利福尼亚大学圣迭戈分校）；Guangdong University of Technology（广东工业大学）；University of Melbourne（墨尔本大学）；Shiga University（滋贺大学）；RIKEN（理化学研究所）
LOCAL: Learning with Orientation Matrix to Infer Causal Structure from Time Series Data
School of Software Engineering, Beijing Jiaotong University（北京交通大学软件学院）；Department of Computer Science, Hong Kong Baptist University（香港浸会大学计算机科学系）；Department of Electrical and Computer Engineering, University of Alberta（阿尔伯塔大学电气与计算机工程系）
Bridging preference-based instrumental variable studies and cluster-randomized encouragement experiments: study design, noncompliance, and average cluster effect ratio
The foundations of cost-sensitive causal classification
Faculty of Economics and Business（经济与商业学院）；Katholieke Universiteit Leuven（鲁文大学）；Data Analytics Laboratory（数据分析实验室）；Solvay Business School（索维亚商学院）；Vrije Universiteit Brussel（布鲁塞尔自由大学）；Department of Applied Mathematics and Theoretical Physics（应用数学与理论物理系）；University of Cambridge（剑桥大学）；School of Economics and Business（经济与商业学院）；University of Chile（智利大学）
School of Mathematics, Physics, and Computing, University of Southern Queensland（数学、物理与计算学院，南方昆士兰大学）；School of Data Science, Lingnan University（数据科学学院，岭南大学）；School of Business, University of Southern Queensland（商学院，南方昆士兰大学）；Schulich School of Engineering, University of Calgary（Schulich工程学院，卡尔加里大学）
