发表机构
Georgia Institute of Technology; Magic Hour（佐治亚理工学院; Magic Hour）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出COPES数据集和三轴评估框架，发现微调能显著提升LLM在心理健康支持中的策略对齐（>50%）和情感对齐，但改进存在异质性且引发分布偏移，偏向问题中心策略。
AI 中文摘要
由于获得专业心理保健的机会仍然有限，许多人转向Reddit等在线平台，寻求基于人类生活经验的同伴支持。然而，此类提问中有相当一部分无人回应，这为使用大型语言模型（LLMs）填补这一空白提供了机会。尽管LLMs在临床基准测试中表现出强劲性能，但其生成基于生活经验且与社区对齐的同伴支持的能力尚未得到充分探索。针对这一空白，我们引入了社区中心同伴参与支持（COPES）数据集和一个三轴评估框架，以评估LLM与社区视角对心理健康支持求助提问的对齐程度。通过评估零样本和训练后（SFT和DPO）模型，我们表明在COPES上进行训练后显著提升了策略对齐（通用模型提升超过50%）以及情感与语气方面的对齐。然而，我们也观察到这些改进是异质的，对齐改进在不同子版块和所请求的应对策略之间存在显著差异。此外，训练后引发了分布偏移，严重偏向以问题为中心的推荐，同时抑制了以情感为中心的策略。总之，这项工作表明，虽然策划社区驱动数据能改善LLM响应的对齐性，但模型性能在不同子社区和特定心理健康需求上仍存在差异。
英文摘要
As access to professional mental healthcare remains limited, many individuals turn to online platforms such as Reddit to seek peer support situated within human lived experience. However, a significant portion of such queries go unanswered, presenting an opportunity for using Large Language Models (LLMs) to fill this gap. While LLMs have demonstrated strong performance on clinical benchmarks, their ability to generate lived-experience informed and community-aligned peer support is underexplored. Addressing this gap, we introduce the COmmunity-centered Peer Engaged Support (COPES) dataset and a three-axis evaluation framework to assess LLM alignment with community perspectives to mental health support seeking queries. Evaluating zero-shot and post-trained (SFT and DPO) models, we show that post-training on COPES significantly improves Strategy Alignment (>50% for general-purpose models) and alignment in Emotion & Tone. However, we also observe that such improvements are heterogeneous and alignment improvements vary significantly across subreddits and requested coping strategies. Furthermore, post-training induces distributional shifts, heavily favoring problem-focused recommendations while suppressing emotion-focused strategies. Together, this work shows that while curating community-driven data improves the alignment of LLM responses, model performance remains disparate across distinct sub-communities and specific mental health needs.
Comments25 pages, 6 figures, 17 tables
