LLM的多彩未来：评估和改进LLM作为酷儿青年的情感支持者
The Colorful Future of LLMs: Evaluating and Improving LLMs as Emotional Supporters for Queer Youth
AI总结：
本文评估了LLM作为酷儿青年情感支持者的潜力，开发了十题量表发现LLM虽具包容性但缺乏个性化，并证明专用提示词可提升性能，提出主动获取上下文的LLM-supporter蓝图以提供个性化可靠回复。
摘要：
酷儿青年面临更高的心理健康风险，如抑郁、焦虑和自杀意念。受负面污名化阻碍，他们常常避免寻求帮助，转而依赖网络资源，而这些资源可能提供不兼容的信息。尽管获得支持性环境和可靠信息是无价的，但全球许多酷儿青年无法获得此类支持。然而，由于 ChatGPT 等大型语言模型（LLM）的快速普及，这种情况可能很快会改变。本文旨在全面探索 LLM 彻底改变酷儿情感支持的潜力。为此，我们对 LLM 与酷儿相关内容的交互进行了定性和定量分析。为了评估回复质量，我们开发了一种受心理学标准和专家意见启发的全新十题量表。我们应用该量表对多个 LLM 和人类在酷儿青年寻求建议及分享经验的帖子下的评论进行评分。我们发现 LLM 的回复具有支持性和包容性，得分超过人类。然而，它们往往过于笼统、共情不足且缺乏个性化，导致建议不可靠且可能有害。我们讨论了这些挑战，证明了专用提示词能提升性能，并提出了一种 LLM-supporter 蓝图，该支持者主动（但敏锐地）获取用户上下文，以提供个性化、共情且可靠的回复。我们的标注数据集可供进一步研究。
英文摘要：
Queer youth face increased mental health risks, such as depression, anxiety, and suicidal ideation. Hindered by negative stigma, they often avoid seeking help and rely on online resources, which may provide incompatible information. Although access to a supportive environment and reliable information is invaluable, many queer youth worldwide have no access to such support. However, this could soon change due to the rapid adoption of Large Language Models (LLMs) such as ChatGPT. This paper aims to comprehensively explore the potential of LLMs to revolutionize emotional support for queers. To this end, we conduct a qualitative and quantitative analysis of LLM's interactions with queer-related content. To evaluate response quality, we develop a novel ten-question scale that is inspired by psychological standards and expert input. We apply this scale to score several LLMs and human comments to posts where queer youth seek advice and share experiences. We find that LLM responses are supportive and inclusive, outscoring humans. However, they tend to be generic, not empathetic enough, and lack personalization, resulting in nonreliable and potentially harmful advice. We discuss these challenges, demonstrate that a dedicated prompt can improve the performance, and propose a blueprint of an LLM-supporter that actively (but sensitively) seeks user context to provide personalized, empathetic, and reliable responses. Our annotated dataset is available for further research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
