社交消息中的零样本叙事检测

中文摘要 AI 辅助

本研究探讨了大型语言模型（LLMs）在识别和分类社交消息中隐藏叙事方面的零样本能力。我们的研究假设是，LLMs广泛的情境知识使其能够在更深层的语用层面上解读消息，超越基本的情绪或主题分析。在Dipromats和SemEval数据集上的实验表明，向模型提供人类撰写的叙事描述能显著提升性能，且无需训练示例。相比之下，自动生成的描述或使用少量示例（少样本）往往因框架的细微偏移而降低准确性。研究还发现，集成方法，尤其是多数投票，能增强鲁棒性，且较大的模型表现最佳，同时对提示变化也较不敏感。研究结果验证了LLMs能在零样本设置下有效检测策略性叙事，并且当与简单集成和人类撰写的描述结合时，可媲美监督系统，为叙事检测提供了一种可扩展的解决方案，尤其是在绝大多数领域缺乏训练数据的情况下。

英文摘要

This study investigates the zero-shot ability of large language models (LLMs) to identify and classify hidden narratives in social messages. Our research hypothesis is that LLMs' extensive contextual knowledge allows them to interpret messages on a deeper, pragmatic level, going beyond basic sentiment or topic analysis. Experiments on the Dipromats and SemEval datasets show that providing models with human-written narrative descriptions significantly improves performance, without the need of training examples. In contrast, automatically generated descriptions or the use of few examples (few-shot) often degrade accuracy due to subtle shifts in framing. The study also finds that ensemble methods, particularly majority voting, enhance robustness and that larger models perform best while also being less sensitive to prompt variations. The findings validate that LLMs can effectively detect strategic narratives in a zero-shot setting, and when combined with simple ensembling and human-written descriptions, they can rival supervised systems, offering a scalable solution for narrative detection, specially when there is no training data for the vast majority of domains.

发表机构

• Universidad Nacional de Educación a Distancia（西班牙国家远程教育大学）
• Zurich University of Applied Sciences（苏黎世应用科学大学）

机构由 AI 辅助整理，请以论文原文为准。