迈向更安全的在线空间:模拟与评估进食障碍讨论的干预策略
Towards Safer Online Spaces: Simulating and Assessing Intervention Strategies for Eating Disorder Discussions
AI总结:
本文提出 LLM 驱动的测试平台,跨平台、模型与主题模拟进食障碍讨论并评估干预策略,发现礼貌型干预效果稳定且模型间存在显著偏差。
摘要:
进食障碍是影响全球数百万人的复杂心理健康疾病。在社交媒体平台上采取有效干预至关重要,但在真实情境中测试策略可能存在风险。我们提出了一种新颖的、由 LLM 驱动的实验测试平台,用于模拟和评估 ED 相关讨论中的干预策略。我们的框架可跨多个平台、模型和 ED 相关主题生成合成对话,从而允许对多种干预方法进行受控实验。我们从四个维度分析各种干预策略对对话动态的影响:干预类型、生成模型、社交媒体平台以及 ED 相关社区/主题。我们采用认知领域分析指标(包括情感、情绪等)来评估干预的有效性。研究结果表明,以礼貌为重点的干预在所有维度上都持续改善了积极情感和情绪基调,而洞察重置方法往往会增加负面情绪。我们还发现 LLM 生成的对话存在显著偏差,认知指标在不同模型之间差异明显(Claude-3 Haiku > Mistral > GPT-3.5-turbo > LLaMA3),甚至同一模型的不同版本之间也是如此。这些差异凸显了模型选择在模拟真实的 ED 相关讨论中的重要性。我们的工作为 ED 相关讨论的复杂动态以及各种干预策略的有效性提供了有价值的信息。
英文摘要:
Eating disorders are complex mental health conditions that affect millions of people around the world. Effective interventions on social media platforms are crucial, yet testing strategies in situ can be risky. We present a novel LLM-driven experimental testbed for simulating and assessing intervention strategies in ED-related discussions. Our framework generates synthetic conversations across multiple platforms, models, and ED-related topics, allowing for controlled experimentation with diverse intervention approaches. We analyze the impact of various intervention strategies on conversation dynamics across four dimensions: intervention type, generative model, social media platform, and ED-related community/topic. We employ cognitive domain analysis metrics, including sentiment, emotions, etc., to evaluate the effectiveness of interventions. Our findings reveal that civility-focused interventions consistently improve positive sentiment and emotional tone across all dimensions, while insight-resetting approaches tend to increase negative emotions. We also uncover significant biases in LLM-generated conversations, with cognitive metrics varying notably between models (Claude-3 Haiku $>$ Mistral $>$ GPT-3.5-turbo $>$ LLaMA3) and even between versions of the same model. These variations highlight the importance of model selection in simulating realistic discussions related to ED. Our work provides valuable information on the complex dynamics of ED-related discussions and the effectiveness of various intervention strategies.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。