arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.16053cs.LGcs.CL
收藏

多目标对齐语言模型以实现个性化心理治疗

Multi-Objective Alignment of Language Models for Personalized Psychotherapy

Department of Computer Science, University of California, Los Angeles, CA, USA(计算机科学系,加州大学洛杉矶分校);Department of Psychology, The University of Texas at Austin, Austin, TX, USA(心理学系,德克萨斯大学奥斯汀分校);Department of Psychiatry, NYU Grossman School of Medicine, New York, NY, USA(精神病学系,纽约大学格罗斯曼医学院)

Mehrab Beikzadeh, Yasaman Asadollah Salmanpour, Ashima Suvarna, Sriram Sankararaman, Matteo Malgaroli, Majid Sarrafzadeh, Saadia Gabriel

AI总结:

本文提出多目标对齐框架,通过直接偏好优化提升语言模型在心理治疗中的共情与安全性能,实验显示其在平衡患者偏好与临床安全方面优于传统方法。

摘要:

全球有超过10亿人受到心理健康障碍的影响,但获取治疗仍然受限于人力资源短缺和成本限制。尽管人工智能系统显示出治疗的潜力,但当前的对齐方法独立优化目标,未能在患者偏好与临床安全之间取得平衡。我们调查了335名具有实际心理健康经验的个体,收集了跨治疗维度的偏好排名,然后开发了一个多目标对齐框架,使用直接偏好优化。我们为六个标准训练奖励模型——共情、安全、积极倾听、自我驱动的改变、信任/共鸣和患者自主性,并系统地比较多目标方法与单目标优化、监督微调和参数合并。多目标DPO(MODPO)在平衡方面表现更优(77.6%共情,62.6%安全)相比单目标优化(93.6%共情,47.8%安全),并且治疗标准在一般沟通原则上表现更优17.2%。盲评临床医生评估确认MODPO始终被偏好,LLM-evaluator的一致性与跨临床医生的可靠性相当。

英文摘要:

Mental health disorders affect over 1 billion people worldwide, yet access to care remains limited by workforce shortages and cost constraints. While AI systems show therapeutic promise, current alignment approaches optimize objectives independently, failing to balance patient preferences with clinical safety. We survey 335 individuals with lived mental health experience to collect preference rankings across therapeutic dimensions, then develop a multi-objective alignment framework using direct preference optimization. We train reward models for six criteria -- empathy, safety, active listening, self-motivated change, trust/rapport, and patient autonomy -- and systematically compare multi-objective approaches against single-objective optimization, supervised fine-tuning, and parameter merging. Multi-objective DPO (MODPO) achieves superior balance (77.6% empathy, 62.6% safety) compared to single-objective optimization (93.6% empathy, 47.8% safety), and therapeutic criteria outperform general communication principles by 17.2%. Blinded clinician evaluation confirms MODPO is consistently preferred, with LLM-evaluator agreement comparable to inter-clinician reliability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑