Data collection from manual labeling provides domain-specific and task-aligned supervision for data-driven approaches, and a critical mass of well-annotated resources is required to achieve reasonable performance in natural language processing tasks. However, manual annotations are often challenging to scale up in terms of time and budget, especially when domain knowledge, capturing subtle semantic features, and reasoning steps are needed. In this paper, we investigate the efficacy of leveraging large language models on automated labeling for computational stance detection. We empirically observe that while large language models show strong potential as an alternative to human annotators, their sensitivity to task-specific instructions and their intrinsic biases pose intriguing yet unique challenges in machine annotation. We introduce a multi-label and multi-target sampling strategy to optimize the annotation quality. Experimental results on the benchmark stance detection corpora show that our method can significantly improve performance and learning efficacy.
Optimizing Social Media Annotation of HPV Vaccine Skepticism and Misinformation Using Large Language Models: An Experimental Evaluation of In-Context Learning and Fine-Tuning Stance Detection Across Multiple Models
School of Journalism and Mass Communication, University of Wisconsin–Madison（威斯康星大学麦迪逊分校新闻与大众传播学院）；Department of Electrical & Computer Engineering, University of Wisconsin–Madison（威斯康星大学麦迪逊分校电气与计算机工程系）；Department of Computer Science, University of Wisconsin–Madison（威斯康星大学麦迪逊分校计算机科学系）；Department of Psychology, University of Wisconsin–Madison（威斯康星大学麦迪逊分校心理学系）；School of Interactive Computing, Georgia Institute of Technology（佐治亚理工学院交互计算学院）；Institute of People and Technology, Georgia Institute of Technology（佐治亚理工学院人与技术研究所）
Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoning
School of Cyber Science and Engineering, Sichuan University（四川大学网络空间安全学院）；School of Cyberspace Science and Technology, Beijing Jiaotong University（北京交通大学网络空间安全学院）
Contextual information integration for stance detection via cross-attention
Technical University of Darmstadt（达姆施塔特工业大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；Lucerne University of Applied Sciences and Arts（卢塞恩应用科学与艺术大学）
How would Stance Detection Techniques Evolve after the Launch of ChatGPT?
College of Big Data and Internet, Shenzhen Technology University（深圳技术大学大数据与互联网学院）；Faculty of Information and Intelligence, Shenzhen X-Institute（深圳X研究所信息与智能学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberation
Xi’an Jiaotong University（西安交通大学）；National University of Singapore（新加坡国立大学）；Yunnan University（云南大学）；Agency for Science, Technology and Research (A*STAR), Singapore（新加坡科技研究局（A*STAR））
Large Language Models Unpack Complex Political Opinions through Target-Stance Extraction
Department of Methodology and Statistics, Utrecht University（方法论与统计学系，乌特雷赫特大学）；Department of Languages, Literature and Communication, Utrecht University（语言、文学与传播系，乌特雷赫特大学）
Acting Flatterers via LLMs Sycophancy: Combating Clickbait with LLMs Opposing-Stance Reasoning
Yangzhou University（扬州大学）；Auburn University（亚伯拉罕大学）；University of South Dakota（南达科他大学）；Institute for Clarity in Documentation（文档清晰研究所）；Inria Paris-Rocquencourt（巴黎-罗quentourt研究所）；Rajiv Gandhi University（拉贾夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
Scaling Public Health Text Annotation: Zero-Shot Learning vs. Crowdsourcing for Improved Efficiency and Labeling Accuracy
University of Toronto（多伦多大学）；Dalla Lana School of Public Health（达拉·拉纳公共卫生学院）；Institute of Health Policy, Management, and Evaluation (IHPME)（卫生政策、管理与评价研究所）；Faculty of Information（信息学院）；Schwartz Reisman Institute（施瓦茨·赖斯曼研究所）
A Context-Aware Dataset for Stance Detection in Bioethical Controversies on Reddit
School of Cyber Science and Technology, University of Science and Technology of China（中国科学技术大学网络空间安全学院）；School of Artificial Intelligence, Shenzhen Technology University（深圳技术大学人工智能学院）；School of Urban Planning and Design, Peking University（北京大学城市规划与设计学院）
Semi-automatic Generation of Multilingual Datasets for Stance Detection in Twitter
Vicomtech Foundation（维康泰克基金会）；Basque Research and Technology Alliance (BRTA)（巴斯克研究与技术联盟（BRTA））；University of the Basque Country UPV/EHU（巴斯克大学（UPV/EHU））
Checkstep Research（Checkstep研究公司）；Sofia University “St. Kliment Ohridski”（索菲亚大学“圣克莱门特·奥赫里德斯基”）；University of Copenhagen（哥本哈根大学）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
