利用预定义的社交噪声构建评估AI模型在虚假信息检测中的性能
Assessing the Performance of AI Models in Misinformation Detection Using Predefined Social Noise Constructs
AI总结：
本研究利用预定义社交噪声构建，评估6种文本模型与1个GNN在虚假信息检测中的性能，发现RoBERTa表现最优，整合社交噪声可提升检测效果。
摘要：
社交媒体平台生成的海量用户原创内容往往会掩盖有意义的讨论，这一现象被概念化为社交噪声。现有虚假信息检测方法主要聚焦于事实核查，却在很大程度上忽略了社交噪声在放大虚假信息方面的作用。本研究利用预定义的社交噪声构建，评估了6种基于文本的模型和1个图神经网络（GNN）在缓解虚假信息影响方面的性能。数据集包含2024年3月至2025年3月俄乌冲突期间，通过X平台的#Ukraine标签收集的50万条推文。先进机器学习模型表现出较强性能，准确率介于79.94%至97.17%之间。RoBERTa表现最优，准确率达97.17%，Cohen's kappa为0.9660，MCC为0.9660，AUC为0.9977；GNN准确率为64.52%，AUC为0.899，通过利用推文间的关系结构，其性能优于随机分类。所有基于文本的模型的kappa值均大于0.75，表明整体一致性较高。使用LIME的可解释AI分析显示，89.6%的预测权重归因于上下文特征而非孤立关键词。研究结果表明，将社交噪声整合到AI流程中可提升虚假信息检测能力，为大规模社交媒体环境下的虚假信息缓解策略提供支持。
英文摘要：
Social media platforms generate vast volumes of user-generated content that often obscure meaningful discourse, a phenomenon conceptualized as social noise. While existing misinformation detection methods primarily focus on factual verification, they largely overlook the role of social noise in amplifying misinformation. This study assesses the performance of six text-based models and a graph neural network (GNN) in mitigating the effects of misinformation using predefined social noise constructs. The dataset consists of 500,000 tweets collected from the X platform using the Ukraine hashtag during the Russia-Ukraine conflict from March 2024 to March 2025. Advanced machine learning models achieve strong performance, with accuracies ranging from 79.94% to 97.17%. RoBERTa demonstrates the highest performance, with an accuracy of 97.17%, Cohen's kappa of 0.9660, MCC of 0.9660, and AUC of 0.9977. The GNN achieves an accuracy of 64.52% and an AUC of 0.899, demonstrating performance above random-chance classification by exploiting the relational structure between tweets. All text-based models achieve kappa values greater than 0.75, indicating substantial overall agreement. Explainable AI analysis using LIME reveals that 89.6% of predictive weight is attributed to contextual features rather than isolated keywords. The findings highlight that integrating social noise into AI pipelines enhances misinformation detection and supports more effective mitigation strategies in large-scale social media environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
