Multi-Objective Alignment of Language Models for Personalized Psychotherapy
Department of Computer Science, University of California, Los Angeles, CA, USA(计算机科学系,加州大学洛杉矶分校);Department of Psychology, The University of Texas at Austin, Austin, TX, USA(心理学系,德克萨斯大学奥斯汀分校);Department of Psychiatry, NYU Grossman School of Medicine, New York, NY, USA(精神病学系,纽约大学格罗斯曼医学院)
Mental health disorders affect over 1 billion people worldwide, yet access to care remains limited by workforce shortages and cost constraints. While AI systems show therapeutic promise, current alignment approaches optimize objectives independently, failing to balance patient preferences with clinical safety. We survey 335 individuals with lived mental health experience to collect preference rankings across therapeutic dimensions, then develop a multi-objective alignment framework using direct preference optimization. We train reward models for six criteria -- empathy, safety, active listening, self-motivated change, trust/rapport, and patient autonomy -- and systematically compare multi-objective approaches against single-objective optimization, supervised fine-tuning, and parameter merging. Multi-objective DPO (MODPO) achieves superior balance (77.6% empathy, 62.6% safety) compared to single-objective optimization (93.6% empathy, 47.8% safety), and therapeutic criteria outperform general communication principles by 17.2%. Blinded clinician evaluation confirms MODPO is consistently preferred, with LLM-evaluator agreement comparable to inter-clinician reliability.
FedPDPO: Federated Personalized Direct Preference Optimization for Large Language Model Alignment
College of Intelligence and Computing, Tianjin University, Tianjin, China(智能与计算学院,天津大学,天津,中国);College of Computing and Data Science, Nanyang Technological University, Singapore(计算与数据科学学院,南洋理工大学,新加坡)
A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models
University of California, San Diego(加利福尼亚大学圣地亚哥分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Adobe Research(奥多比研究院);The Ohio State University(俄亥俄州立大学);Allen Institute for AI(艾伦人工智能研究所)
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications
Ant Group(蚂蚁集团);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院)
One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment
The Hong Kong Polytechnic University(香港理工大学);Huawei Technologies Ltd.(华为技术有限公司);National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学)
MAPS: Motivation-Aware Personalized Search via LLM-Driven Consultation Alignment
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);University of International Business and Economics(国际商务经济大学);AI Lab at Lenovo Research, Lenovo Group Limited(联想集团研究院人工智能实验室)
From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS(中国科学院自动化研究所多模态人工智能系统国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
BAPO: Base-Anchored Preference Optimization for Overcoming Forgetting in Large Language Models Personalization
Graduate School of AI, KAIST(韩国科学技术院人工智能研究生院);Samsung Advanced Institute of Technology(三星先进技术研究院);Department of Electrical and Computer Engineering, UBC(不列颠哥伦比亚大学电气与计算机工程系)
COMPASS: Computational Mapping of Patient-Therapist Alliance Strategies with Language Modeling
Department of Artificial Intelligence and Human Health, Icahn School of Medicine at Mount Sinai(人工智能与人类健康系,伊坎医学院 Mount Sinai 分校);Department of Psychiatry, Icahn School of Medicine at Mount Sinai(精神病学系,伊坎医学院 Mount Sinai 分校);Department of Neuroscience, Icahn School of Medicine at Mount Sinai(神经科学系,伊坎医学院 Mount Sinai 分校);Berkman Klein Center for Internet & Society, Harvard University(互联网与社会研究中心,哈佛大学);IBM Research, T.J. Watson Research Center(IBM 研究,T.J. Watson 研究中心);Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center(精神疾病研究、教育与临床中心,James J. Peters VA 医疗中心)
EduAgentQG: Multi-Agent Personalized Mathematics Question Generation with Explicit Diversity and Objective-Aware Evaluation
East China Normal University(东华大学);Shanghai Institute of Al for Education(上海人工智能教育研究院);The Chinese University of Hong Kong, Shenzhen School of Data Science(香港中文大学(深圳)数据科学学院)
"In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Ant Group(蚂蚁集团);University of Electronic Science and Technology of China(电子科技大学);Kunming University of Science and Technology(昆明理工大学)
Large Language Models Empowered Personalized Web Agents
National University of Singapore(新加坡国立大学);The Hong Kong Polytechnic University(香港理工大学);University of Science and Technology of China(中国科学技术大学);Eastern Institute of Technology(宁波东方理工大学)
From Critique to Clarity: A Pathway to Faithful and Personalized Code Explanations with Large Language Models
Institute for Clarity in Documentation(文献清晰度研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康库尔中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室);The Thørväld Group(索尔沃尔德集团);The Kumquat Consortium(金桔联盟)