Stance Detection (StD) aims to detect an author's stance towards a certain topic or claim and has become a key component in applications like fake news detection, claim validation, and argument search. However, while stance is easily detected by humans, machine learning models are clearly falling short of this task. Given the major differences in dataset sizes and framing of StD (e.g. number of classes and inputs), we introduce a StD benchmark that learns from ten StD datasets of various domains in a multi-dataset learning (MDL) setting, as well as from related tasks via transfer learning. Within this benchmark setup, we are able to present new state-of-the-art results on five of the datasets. Yet, the models still perform well below human capabilities and even simple adversarial attacks severely hurt the performance of MDL models. Deeper investigation into this phenomenon suggests the existence of biases inherited from multiple datasets by design. Our analysis emphasizes the need of focus on robustness and de-biasing strategies in multi-task learning approaches. The benchmark dataset and code is made available.
StanceFlip: A Comprehensive Multi-Dimensional Benchmark for Multimodal Conversational Stance Flipping Forecasting
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件学院）；Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳））；City University of Macau（澳门城市大学）
DimStance: Multilingual Datasets for Dimensional Stance Analysis
University of Göttingen（弗赖堡大学）；Yuan Ze University（元智大学）；Imperial College London（伦敦帝国理工学院）；University of Pretoria（南非普里蒂亚大学）；National Yang Ming Chiao Tung University；Maseno University（马斯诺大学）；Central Police University Taiwan（台湾中央警察大学）；Yunnan University（云南大学）；Bayero University Kano（卡诺巴耶罗大学）；National Research Council Canada（加拿大国家研究委员会）
Multi-modal Stance Detection: New Datasets and Model
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东省新型安全智能技术重点实验室）；King’s College London（伦敦国王学院）；SIAT, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Peng Cheng Laboratory（鹏城实验室）；The Chinese University of Hong Kong（香港中文大学）
Read Between the Lines: A Benchmark for Uncovering Political Bias in Bangla News Articles
University of Dhaka（达卡大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Cisco Systems（思科系统）；BanglaLLM；Maharishi International University（玛哈里希国际大学）；Unityflow AI
Qatar Computing Research Institute, HBKU（卡塔尔计算研究所（哈马德·本·哈利法大学））；Department of Computer Science, Columbia University（哥伦比亚大学计算机科学系）；Georgia Institute of Technology（佐治亚理工学院）；The State University of New York at Buffalo（纽约州立大学布法罗分校）；The University of British Columbia（英属哥伦比亚大学）
With the continuing spread of misinformation and disinformation online, it is of increasing importance to develop combating mechanisms at scale in the form of automated systems that support multiple languages. One task of interest is claim veracity prediction, which can be addressed using stance detection with respect to relevant documents retrieved online. To this end, we present our new Arabic Stance Detection dataset (AraStance) of 4,063 claim--article pairs from a diverse set of sources comprising three fact-checking websites and one news website. AraStance covers false and true claims fro…
Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance
Indiana University Bloomington（印第安纳大学伯明顿分校）；Center for Complex Networks and Systems（复杂网络与系统中心）
Zero-Shot Conversational Stance Detection: Dataset and Approaches
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University（航空信息安全与可信计算重点实验室、教育部、网络安全科学与工程学院、武汉大学）
Southeast University（东南大学）；University of Warwick（华威大学）
Fact verification is a challenging task that requires simultaneously reasoning and aggregating over multiple retrieved pieces of evidence to evaluate the truthfulness of a claim. Existing approaches typically (i) explore the semantic interaction between the claim and evidence at different granularity levels but fail to capture their topical consistency during the reasoning process, which we believe is crucial for verification; (ii) aggregate multiple pieces of evidence equally without considering their implicit stances to the claim, thereby introducing spurious information. To alleviate the ab…
Information Sciences Institute, University of Southern California（南加州大学信息科学研究所）；University of Southern California（南加州大学）
Stance detection infers a text author's attitude towards a target. This is challenging when the model lacks background knowledge about the target. Here, we show how background knowledge from Wikipedia can help enhance the performance on stance detection. We introduce Wikipedia Stance Detection BERT (WS-BERT) that infuses the knowledge into stance encoding. Extensive results on three benchmark datasets covering social media discussions and online debates indicate that our model significantly outperforms the state-of-the-art methods on target-specific stance detection, cross-target stance detect…
Zhejiang University（浙江大学）；Westlake University（西湖大学）；Westlake Institute for Advanced Study（西湖高等研究院）
The stance detection task aims to classify the stance toward given documents and topics. Since the topics can be implicit in documents and unseen in training data for zero-shot settings, we propose to boost the transferability of the stance detection model by using sentiment and commonsense knowledge, which are seldom considered in previous studies. Our model includes a graph autoencoder module to obtain commonsense knowledge and a stance detection module with sentiment and commonsense. Experimental results show that our model outperforms the state-of-the-art methods on the zero-shot and few-s…
Contextual information integration for stance detection via cross-attention
Technical University of Darmstadt（达姆施塔特工业大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；Lucerne University of Applied Sciences and Arts（卢塞恩应用科学与艺术大学）
Hangzhou Dianzi University（杭州电子科技大学）；University of Yamanashi（山梨大学）
Scientific claim verification can help the researchers to easily find the target scientific papers with the sentence evidence from a large corpus for the given claim. Some existing works propose pipeline models on the three tasks of abstract retrieval, rationale selection and stance prediction. Such works have the problems of error propagation among the modules in the pipeline and lack of sharing valuable information among modules. We thus propose an approach, named as ARSJoint, that jointly learns the modules for the three tasks with a machine reading comprehension framework by including clai…
Rationale-Guided Knowledge Distillation for Cross-Lingual Stance Detection
School of Foreign Studies, Hefei University of Technology（合肥工业大学外国语学院）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）；Hefei Institute of Technology（合肥学院）
Vaccine hesitancy is considered as one main cause of the stagnant uptake ratio of COVID-19 vaccines in Europe and the US where vaccines are sufficiently supplied. Fast and accurate grasp of public attitudes toward vaccination is critical to address vaccine hesitancy, and social media platforms have proved to be an effective source of public opinions. In this paper, we describe the collection and release of a dataset of tweets related to COVID-19 vaccines. This dataset consists of the IDs of 2,198,090 tweets collected from Western Europe, 17,934 of which are annotated with the originators' vacc…
Stance detection has emerged as an area of research in the field of artificial intelligence. However, most research is currently centered on the target-dependent stance detection task, which is based on a person's stance in favor of or against a specific target. Furthermore, most benchmark datasets are based on English, making it difficult to develop models in low-resource languages such as Korean, especially for an emerging field such as stance detection. This study proposes the LArge-Scale Target-Independent STance (LASTIST) dataset to fill this research gap. Collected from the press release…
University of Michigan（密歇根大学）；Northeastern University（东北大学）
Ideology is at the core of political science research. Yet, there still does not exist general-purpose tools to characterize and predict ideology across different genres of text. To this end, we study Pretrained Language Models using novel ideology-driven pretraining objectives that rely on the comparison of articles on the same story written by media of different ideologies. We further collect a large-scale dataset, consisting of more than 3.6M political news articles, for pretraining. Our model POLITICS outperforms strong baselines and the previous state-of-the-art models on ideology predict…
Zero-shot stance detection is challenging because it requires detecting the stance of previously unseen targets in the inference phase. The ability to learn transferable target-invariant features is critical for zero-shot stance detection. In this work, we propose a stance detection approach that can efficiently adapt to unseen targets, the core of which is to capture target-invariant syntactic expression patterns as transferable knowledge. Specifically, we first augment the data by masking the topic words of sentences, and then feed the augmented data to an unsupervised contrastive learning m…
As social media becomes a hotbed for the spread of misinformation, the crucial task of rumor detection has witnessed promising advances fostered by open-source benchmark datasets. Despite being widely used, we find that these datasets suffer from spurious correlations, which are ignored by existing studies and lead to severe overestimation of existing rumor detection performance. The spurious correlations stem from three causes: (1) event-based data collection and labeling schemes assign the same veracity label to multiple highly similar posts from the same underlying event; (2) merging multip…
Stance detection on social media is an emerging opinion mining paradigm for various social and political applications in which sentiment analysis may be sub-optimal. There has been a growing research interest for developing effective methods for stance detection methods varying among multiple communities including natural language processing, web science, and social computing. This paper surveys the work on stance detection within those communities and situates its usage within current opinion mining techniques in social media. It presents an exhaustive review of stance detection techniques on…
Contextualized Prompting For Stance Detection On Social Media
Institute of Intensive Care, University Hospital of Zurich and University of Zurich（重症医学研究所，苏黎世大学医院和苏黎世大学）；Institute of Computer Science, University of Goettingen（计算机科学研究所，哥廷根大学）；GESIS Leibniz Institute for the Social Sciences（社会科学研究莱比锡研究所）；Institut für Publizistik, Johannes Gutenberg-University Mainz（主笔研究所，美因茨约翰· Gutenberg 大学）；Ubiquitous Knowledge Processing Lab, Technical University of Darmstadt（无处不在知识处理实验室，达姆施塔特技术大学）
Hong Kong Baptist University（香港浸会大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The University of Hong Kong（香港大学）
Target-specific stance detection on social media, which aims at classifying a textual data instance such as a post or a comment into a stance class of a target issue, has become an emerging opinion mining paradigm of importance. An example application would be to overcome vaccine hesitancy in combating the coronavirus pandemic. However, existing stance detection strategies rely merely on the individual instances which cannot always capture the expressed stance of a given target. In response, we address a new task called conversational stance detection which is to infer the stance towards a giv…
Collaborative Stance Detection via Small-Large Language Model Consistency Verification
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；People Public Security University of China（中国人民公安大学）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Laboratory（中关村实验室）
Robust 2004 is an information retrieval benchmark whose large number of judgments per query make it a reliable evaluation dataset. In this paper, we present mRobust04, a multilingual version of Robust04 that was translated to 8 languages using Google Translate. We also provide results of three different multilingual retrievers on this dataset. The dataset is available at https://huggingface.co/datasets/unicamp-dl/mrobust
