提升和评估大型语言模型与在线社区对齐的保真度
Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities
AI总结：
针对大型语言模型与特定人群对齐及保真度评估的挑战，本文提出指令微调对齐框架并从多语言维度评估，应用于节食和身体意象社区，通过心理测量量表揭示风险，为自动化内容审核等提供支持。
摘要：
大型语言模型（LLMs）在表征个体和社区方面展现出潜力，为研究复杂社会动态提供了新方法。然而，有效将LLMs与特定人群对齐并系统评估对齐保真度仍是挑战。本文提出一个稳健框架，通过指令微调使LLMs与在线社区对齐，并从语言的真实性、情感基调、毒性和伤害性等多方面综合评估对齐效果。我们将该方法应用于以节食和身体意象为中心的在线社区，证明其效用。通过向对齐后的LLMs施测进食障碍心理测量量表，揭示其不健康信念，并成功区分进食障碍风险水平不同的社区。研究结果凸显了LLMs在自动化 moderation（内容审核）及公共卫生、社会科学研究等更广泛领域的应用潜力。
英文摘要：
Large language models (LLMs) have shown promise in representing individuals and communities, offering new ways to study complex social dynamics. However, effectively aligning LLMs with specific human groups and systematically assessing the fidelity of the alignment remains a challenge. This paper presents a robust framework for aligning LLMs with online communities via instruction-tuning and comprehensively evaluating alignment across various aspects of language, including authenticity, emotional tone, toxicity, and harm. We demonstrate the utility of our approach by applying it to online communities centered on dieting and body image. We administer an eating disorder psychometric test to the aligned LLMs to reveal unhealthy beliefs and successfully differentiate communities with varying levels of eating disorder risk. Our results highlight the potential of LLMs in automated moderation and broader applications in public health and social science research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
