arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.04750cs.CLcs.AI

利用上下文信息提升LLMs在非正式政治 discourse中的立场检测

Exploiting contextual information to improve stance detection in informal political discourse with LLMs

Khoury College of Computer Sciences Northeastern University(科里学院计算机科学系东北大学)

Arman Engin Sucu, Yixiang Zhou, Mario A. Nascimento, Tony Mullen

AI总结:

本研究通过引入用户资料上下文信息,利用LLMs提升非正式政治讨论中立场检测的准确性,实验表明上下文提示可使准确率提升17.5%-38.5%。

摘要:

本研究探讨了使用大型语言模型(LLMs)对非正式在线讨论中的政治立场进行检测,其中语言往往充满讽刺、模糊和依赖上下文。我们探讨了提供上下文信息,特别是从历史帖子中提取的用户资料摘要,是否能提高分类准确性。使用一个现实世界的政治论坛数据集,我们生成了结构化的资料,总结了用户的意识形态倾向、反复出现的主题和语言模式。我们通过全面的跨模型评估,评估了七种最先进的LLMs在基准和上下文增强设置中的表现。我们的发现表明,上下文提示显著提高了准确性,改进幅度从+17.5%到+38.5%,达到74%的准确性,超过了以往的方法。我们还分析了资料大小和帖子选择策略对性能的影响,表明战略性选择的政治内容比更大、随机选择的上下文效果更好。这些发现强调了在复杂的政治分类任务中纳入用户层面上下文的价值。

英文摘要:

This study investigates the use of Large Language Models (LLMs) for political stance detection in informal online discourse, where language is often sarcastic, ambiguous, and context-dependent. We explore whether providing contextual information, specifically user profile summaries derived from historical posts, can improve classification accuracy. Using a real-world political forum dataset, we generate structured profiles that summarize users' ideological leaning, recurring topics, and linguistic patterns. We evaluate seven state-of-the-art LLMs across baseline and context-enriched setups through a comprehensive cross-model evaluation. Our findings show that contextual prompts significantly boost accuracy, with improvements ranging from +17.5\% to +38.5\%, achieving up to 74\% accuracy that surpasses previous approaches. We also analyze how profile size and post selection strategies affect performance, showing that strategically chosen political content yields better results than larger, randomly selected contexts. These findings underscore the value of incorporating user-level context to enhance LLM performance in nuanced political classification tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑