开放权重大语言模型是否准备好用于社交媒体审查？对Bluesky的比较研究
Are Open-Weight LLMs Ready for Social Media Moderation? A Comparative Study on Bluesky
AI总结：
本文评估了开放权重LLMs在社交媒体审查中的表现，发现其在敏感性和特异性上与专有模型有较高重叠，并探讨了在不同审查场景下的应用考量。
摘要：
随着互联网接入的扩大，有害内容的暴露也随之增加，从而增加了有效审查的需求。研究表明，大型语言模型（LLMs）可以有效用于社交媒体审查任务，包括有害内容检测。虽然专有LLMs已被证明在零样本情况下优于传统机器学习模型，但开放权重LLMs的即开能力仍是一个开放性问题。受最近推理LLMs的发展启发，我们评估了七种最先进的模型：四种专有模型和三种开放权重模型。通过在Bluesky上的真实帖子进行测试，审查服务Bluesky的审查决定以及两位作者的注释，我们发现开放权重LLMs的灵敏度（81% - 97%）和特异性（91% - 100%）与专有模型（72% - 98%，93% - 99%）之间有相当大的重叠。此外，我们的分析表明，对于粗鲁检测，特异性超过灵敏度，但对不耐受和威胁则相反。最后，我们识别了人类审查员和LLMs之间的评分一致，突显了在平台规模和个性化审查情境中部署LLMs的考虑因素。这些发现表明，开放权重LLMs可以在消费级硬件上支持隐私保护的审查，并提出了新的方向，用于设计平衡社区价值观与个人用户偏好的审查系统。
英文摘要：
As internet access expands, so does exposure to harmful content, increasing the need for effective moderation. Research has demonstrated that large language models (LLMs) can be effectively utilized for social media moderation tasks, including harmful content detection. While proprietary LLMs have been shown to zero-shot outperform traditional machine learning models, the out-of-the-box capability of open-weight LLMs remains an open question. Motivated by recent developments of reasoning LLMs, we evaluate seven state-of-the-art models: four proprietary and three open-weight. Testing with real-world posts on Bluesky, moderation decisions by Bluesky Moderation Service, and annotations by two authors, we find a considerable degree of overlap between the sensitivity (81%--97%) and specificity (91%--100%) of the open-weight LLMs and those (72%--98%, and 93%--99%) of the proprietary ones. Additionally, our analysis reveals that specificity exceeds sensitivity for rudeness detection, but the opposite holds for intolerance and threats. Lastly, we identify inter-rater agreement across human moderators and the LLMs, highlighting considerations for deploying LLMs in both platform-scale and personalized moderation contexts. These findings show open-weight LLMs can support privacy-preserving moderation on consumer-grade hardware and suggest new directions for designing moderation systems that balance community values with individual user preferences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
