Subjective NLP tasks usually rely on human annotations provided by multiple annotators, whose judgments may vary due to their diverse backgrounds and life experiences. Traditional methods often aggregate multiple annotations into a single ground truth, disregarding the diversity in perspectives that arises from annotator disagreement. In this preliminary study, we examine the effect of including multiple annotations on model accuracy in classification. Our methodology investigates the performance of perspective-aware classification models in stance detection task and further inspects if annotator disagreement affects the model confidence. The results show that multi-perspective approach yields better classification performance outperforming the baseline which uses the single label. This entails that designing more inclusive perspective-aware AI models is not only an essential first step in implementing responsible and ethical AI, but it can also achieve superior results than using the traditional approaches.
Cross-target Stance Detection by Exploiting Target Analytical Perspectives
Shenzhen University(深圳大学);Shenzhen X-Institute(深圳X-Institute);Shenzhen Technology University(深圳技术大学);Harbin Institute of Technology(哈尔滨工业大学);Aoyama Gakuin University(青山学院大学);South China Agricultural University(华南农业大学)
A Challenge Dataset and Effective Models for Conversational Stance Detection
Shenzhen University(深圳大学);Shenzhen Technology University(深圳技术大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳))
Collaborative Knowledge Infusion for Low-resource Stance Detection
Centre for Frontier AI Research (CFAR)(前沿人工智能研究中心(CFAR));Institute of High Performance Computing (IHPC)(高性能计算研究所(IHPC));Agency for Science Technology and Research (A*STAR)(新加坡科技研究局(A*STAR))
Mitigating Biases of Large Language Models in Stance Detection with Counterfactual Augmented Calibration
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Peng Cheng Laboratory(鹏城实验室);The Chinese University of Hong Kong(香港中文大学);King’s College London(伦敦国王学院);The 30th Research Institute of China Electronics Technology Group Corporation(中国电子科技集团公司第三十研究所)
Multi-modal Stance Detection: New Datasets and Model
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies(广东省新型安全智能技术重点实验室);King’s College London(伦敦国王学院);SIAT, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Peng Cheng Laboratory(鹏城实验室);The Chinese University of Hong Kong(香港中文大学)
Rethinking stance detection: A theoretically-informed research agenda for user-level inference using language models
Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)(高性能计算研究所(A*STAR));School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算与信息系统学院);The Hong Kong Polytechnic University(香港理工大学)
Zero-Shot Stance Detection in the Wild: Dynamic Target Generation and Multi-Target Adaptation
School of Information Engineering, Minzu University of China(中国民族大学信息工程学院);National Language Resource Monitoring and Research Center of Minority Languages, Beijing(少数民族语言资源监测与研究中心);Institute of National Security, Minzu University of China(中国民族大学国家安全学院);School of Minority Languages and Literatures, Minzu University of China(中国民族大学少数民族语言文学学院)
Contextual information integration for stance detection via cross-attention
Technical University of Darmstadt(达姆施塔特工业大学);Hessian Center for AI (hessian.AI)(黑森人工智能中心);Lucerne University of Applied Sciences and Arts(卢塞恩应用科学与艺术大学)
MT2-CSD: A New Dataset and Multi-Semantic Knowledge Fusion Method for Conversational Stance Detection
College of Big Data and Internet, Shenzhen Technology University(大数据与互联网学院,深圳技术大学);University of Washington(华盛顿大学);University of Science and Technology of China(中国科学技术大学)
How would Stance Detection Techniques Evolve after the Launch of ChatGPT?
College of Big Data and Internet, Shenzhen Technology University(深圳技术大学大数据与互联网学院);Faculty of Information and Intelligence, Shenzhen X-Institute(深圳X研究所信息与智能学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
AraStance: A Multi-Country and Multi-Domain Dataset of Arabic Stance Detection for Fact Checking
Qatar Computing Research Institute, HBKU(卡塔尔计算研究所(哈马德·本·哈利法大学));Department of Computer Science, Columbia University(哥伦比亚大学计算机科学系);Georgia Institute of Technology(佐治亚理工学院);The State University of New York at Buffalo(纽约州立大学布法罗分校);The University of British Columbia(英属哥伦比亚大学)
Rationale-Guided Knowledge Distillation for Cross-Lingual Stance Detection
School of Foreign Studies, Hefei University of Technology(合肥工业大学外国语学院);School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机与信息工程学院);Hefei Institute of Technology(合肥学院)