发表机构
University of Bucharest（布加勒斯特大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文通过比较掩码与因果语言模型，在SemEval-2020 Task 11上评估宣传技巧检测，最佳模型F1达63.62，并提出微调与集成可提升性能。
AI 中文摘要
宣传检测是自然语言处理（NLP）中的一项重要任务，尤其是在操纵性政治传播的背景下。然而，识别具体的宣传技巧因其往往微妙且依赖上下文而构成重大挑战，使其难以与合法的说服性语言区分开来。宣传常常涉及突出某些事实，同时淡化或忽略其他事实，以制造期望的认知。这种有偏见的传播旨在影响人们对特定事业或立场的态度、信念或行为。本文通过使用SemEval-2020 Task 11数据集，对现代语言模型进行比较分析，探索宣传技巧检测的进展。我们评估了掩码语言模型（基于XLM-RoBERTa或DeBERTa V3）和因果模型（来自OpenAI、Google、Mistral、Anthropic和Meta），采用了两种提示策略：基础提示和思维链提示。我们的结果显示，相较于最先进的模型有所改进，表现最佳的MLM在技巧分类中取得了63.18的F1分数，而最佳因果模型达到了63.62。我们还观察到，某些模型在特定技巧上表现出色，如加载语言和指名道姓，而在其他技巧如跟风和黑白谬误上则表现不佳。这些发现表明，微调、集成建模以及使用更大的数据集可以进一步增强宣传检测能力。
英文摘要
Propaganda detection is an essential task in natural language processing (NLP), particularly in the context of manipulative political communications. However, identifying specific propaganda techniques presents a significant challenge due to their often subtle nature and reliance on context, making them difficult to distinguish from legitimate persuasive language. Propaganda often involves highlighting certain facts while downplaying or ignoring others to create a desired perception. This biased communication aims to influence attitudes, beliefs, or behaviors towards a particular cause or position. This paper explores advances in detecting propaganda techniques through a comparative analysis of modern language models, using the SemEval-2020 Task 11 dataset. We evaluated both masked language models (based on XLM-RoBERTa or DeBERTa V3) and causal models (from OpenAI, Google, Mistral, Anthropic and Meta), employing two prompting strategies: base and chain-of-thought prompting. Our results demonstrate improvements over state-of-the-art models, with the best-performing MLM achieving an F1 score of 63.18 in technique classification and the best causal model achieving 63.62. We also observed that certain models excel in specific techniques, such as loaded language and name-calling, while struggling with others like bandwagon and black-and-white fallacy. These findings suggest that fine-tuning, ensemble modeling, and the use of larger datasets can further enhance propaganda detection capabilities.
