Zero-shot stance detection is challenging because it requires detecting the stance of previously unseen targets in the inference phase. The ability to learn transferable target-invariant features is critical for zero-shot stance detection. In this work, we propose a stance detection approach that can efficiently adapt to unseen targets, the core of which is to capture target-invariant syntactic expression patterns as transferable knowledge. Specifically, we first augment the data by masking the topic words of sentences, and then feed the augmented data to an unsupervised contrastive learning module to capture transferable features. Then, to fit a specific target, we encode the raw texts as target-specific features. Finally, we adopt an attention mechanism, which combines syntactic expression patterns with target-specific features to obtain enhanced features for predicting previously unseen targets. Experiments demonstrate that our model outperforms competitive baselines on four benchmark datasets.
The impressive performance of GPT-3 using natural language prompts and in-context learning has inspired work on better fine-tuning of moderately-sized models under this paradigm. Following this line of work, we present a contrastive learning framework that clusters inputs from the same class for better generality of models trained with only limited examples. Specifically, we propose a supervised contrastive framework that clusters inputs from the same class under different augmented "views" and repel the ones from different classes. We create different "views" of an example by appending it wit…
Pohang University of Science and Technology (POSTECH)（浦项科技大学）
Zero-shot slot filling has received considerable attention to cope with the problem of limited available data for the target domain. One of the important factors in zero-shot learning is to make the model learn generalized and reliable representations. For this purpose, we present mcBERT, which stands for momentum contrastive learning with BERT, to develop a robust zero-shot slot filling model. mcBERT uses BERT to initialize the two encoders, the query encoder and key encoder, and is trained by applying momentum contrastive learning. Our experimental results on the SNIPS benchmark show that mc…
Dalian University of Technology（大连理工大学）；Peking University（北京大学）
Intent classification (IC) and slot filling (SF) are critical building blocks in task-oriented dialogue systems. These two tasks are closely-related and can flourish each other. Since only a few utterances can be utilized for identifying fast-emerging new intents and slots, data scarcity issue often occurs when implementing IC and SF. However, few IC/SF models perform well when the number of training samples per class is quite small. In this paper, we propose a novel explicit-joint and supervised-contrastive learning framework for few-shot intent classification and slot filling. Its highlights…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
In this paper, we explore how to utilize pre-trained language model to perform few-shot text classification where only a few annotated examples are given for each class. Since using traditional cross-entropy loss to fine-tune language model under this scenario causes serious overfitting and leads to sub-optimal generalization of model, we adopt supervised contrastive learning on few labeled data and consistency-regularization on vast unlabeled data. Moreover, we propose a novel contrastive consistency to further boost model performance and refine sentence representation. After conducting exten…
Pre-trained Language Models (PLMs) have achieved remarkable performance for various language understanding tasks in IR systems, which require the fine-tuning process based on labeled training data. For low-resource scenarios, prompt-based learning for PLMs exploits prompts as task guidance and turns downstream tasks into masked language problems for effective few-shot fine-tuning. In most existing approaches, the high performance of prompt-based learning heavily relies on handcrafted prompts and verbalizers, which may limit the application of such approaches in real-world scenarios. To solve t…
School of Information；Communications Xi’an Jiaotong University Xi'an, P.R. China
Most previous few-shot learning algorithms are based on meta-training with fake few-shot tasks as training samples, where large labeled base classes are required. The trained model is also limited by the type of tasks. In this paper we propose a new paradigm of unsupervised few-shot learning to repair the deficiencies. We solve the few-shot tasks in two phases: meta-training a transferable feature extractor via contrastive self-supervised learning and training a classifier using graph aggregation, self-distillation and manifold augmentation. Once meta-trained, the model can be used in any type…
Dalian University of Technology（大连理工大学）；Peking University（北京大学）
Multi-label aspect category detection allows a given review sentence to contain multiple aspect categories, which is shown to be more practical in sentiment analysis and attracting increasing attention. As annotating large amounts of data is time-consuming and labor-intensive, data scarcity occurs frequently in real-world scenarios, which motivates multi-label few-shot aspect category detection. However, research on this problem is still in infancy and few methods are available. In this paper, we propose a novel label-enhanced prototypical network (LPN) for multi-label few-shot aspect category…
Tsinghua University（清华大学）；Tencent Youtu Lab（腾讯优图实验室）；University College London（伦敦大学学院）
Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples in support images. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. However, this framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only. To address this issue, in this paper, we introduce a complementary self-contrastive task into few-shot semantic segmentation. Our new model is able to associate the pi…
CoSD: Collaborative Stance Detection with Contrastive Heterogeneous Topic Graph Learning
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Department of Computer Science, Tongji University（同济大学计算机科学系）；College of Computer Science and Technology, Taiyuan University of Technology（太原理工大学计算机科学与技术学院）
Contrastive self-supervised learning methods learn to map data points such as images into non-parametric representation space without requiring labels. While highly successful, current methods require a large amount of data in the training phase. In situations where the target training set is limited in size, generalization is known to be poor. Pretraining on a large source data set and fine-tuning on the target samples is prone to overfitting in the few-shot regime, where only a small number of target samples are available. Motivated by this, we propose a domain adaption method for self-super…
MIT Computer Science and Artificial Intelligence Laboratory（麻省理工学院计算机科学与人工智能实验室）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
We study cross-lingual stance detection, which aims to leverage labeled data in one language to identify the relative perspective (or stance) of a given document with respect to a claim in a different target language. In particular, we introduce a novel contrastive language adaptation approach applied to memory networks, which ensures accurate alignment of stances in the source and target languages, and can effectively deal with the challenge of limited labeled data in the target language. The evaluation results on public benchmark datasets and comparison against current state-of-the-art appro…
Stance detection is an important component of understanding hidden influences in everyday life. Since there are thousands of potential topics to take a stance on, most with little to no training data, we focus on zero-shot stance detection: classifying stance from no training examples. In this paper, we present a new dataset for zero-shot stance detection that captures a wider range of topics and lexical variation than in previous datasets. Additionally, we propose a new model for stance detection that implicitly captures relationships between topics using generalized topic representations and…
Stance detection on social media can help to identify and understand slanted news or commentary in everyday life. In this work, we propose a new model for zero-shot stance detection on Twitter that uses adversarial learning to generalize across topics. Our model achieves state-of-the-art performance on a number of unseen test topics with minimal computational costs. In addition, we extend zero-shot stance detection to new topics, highlighting future directions for zero-shot transfer.
Huazhong University of Science and Technology（华中科技大学）
The rapid development of Industry 4.0 has amplified the scope and destructiveness of industrial Cyber-Physical System (CPS) by network attacks. Anomaly detection techniques are employed to identify these attacks and guarantee the normal operation of industrial CPS. However, it is still a challenging problem to cope with scenarios with few labeled samples. In this paper, we propose a few-shot anomaly detection model (FSL-PN) based on prototypical network and contrastive learning for identifying anomalies with limited labeled data from industrial CPS. Specifically, we design a contrastive loss t…
Rethinking the Sample Relations for Few-Shot Classification
School of Big Data and Software Engineering, Chongqing University（重庆大学大数据与软件学院）；Chongqing University（重庆大学）；Fowler College of Business, San Diego State University（圣地亚哥州立大学福勒商学院）；San Diego State University（圣地亚哥州立大学）
Zhejiang University（浙江大学）；Westlake University（西湖大学）；Westlake Institute for Advanced Study（西湖高等研究院）
The stance detection task aims to classify the stance toward given documents and topics. Since the topics can be implicit in documents and unseen in training data for zero-shot settings, we propose to boost the transferability of the stance detection model by using sentiment and commonsense knowledge, which are seldom considered in previous studies. Our model includes a graph autoencoder module to obtain commonsense knowledge and a stance detection module with sentiment and commonsense. Experimental results show that our model outperforms the state-of-the-art methods on the zero-shot and few-s…
Prior studies of zero-shot stance detection identify the attitude of texts towards unseen topics occurring in the same document corpus. Such task formulation has three limitations: (i) Single domain/dataset. A system is optimized on a particular dataset from a single domain; therefore, the resulting system cannot work well on other datasets; (ii) the model is evaluated on a limited number of unseen topics; (iii) it is assumed that part of the topics has rich annotations, which might be impossible in real-world applications. These drawbacks will lead to an impractical stance detection system th…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Stance detection aims to identify whether the author of a text is in favor of, against, or neutral to a given target. The main challenge of this task comes two-fold: few-shot learning resulting from the varying targets and the lack of contextual information of the targets. Existing works mainly focus on solving the second issue by designing attention-based models or introducing noisy external knowledge, while the first issue remains under-explored. In this paper, inspired by the potential capability of pre-trained language models (PLMs) serving as knowledge bases and few-shot learners, we prop…
Zero-Shot Conversational Stance Detection: Dataset and Approaches
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University（航空信息安全与可信计算重点实验室、教育部、网络安全科学与工程学院、武汉大学）
Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance
Indiana University Bloomington（印第安纳大学伯明顿分校）；Center for Complex Networks and Systems（复杂网络与系统中心）
Zero-Shot Stance Detection in the Wild: Dynamic Target Generation and Multi-Target Adaptation
School of Information Engineering, Minzu University of China（中国民族大学信息工程学院）；National Language Resource Monitoring and Research Center of Minority Languages, Beijing（少数民族语言资源监测与研究中心）；Institute of National Security, Minzu University of China（中国民族大学国家安全学院）；School of Minority Languages and Literatures, Minzu University of China（中国民族大学少数民族语言文学学院）
FarExStance: Explainable Stance Detection for Farsi
Dublin City University（都柏林城市大学）；Iran University of Science and Technology（伊朗科技大学）；Dadmatech（Dadmatech公司）；University of Birmingham（伯明翰大学）；Cardiff University（卡迪夫大学）
Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoning
School of Cyber Science and Engineering, Sichuan University（四川大学网络空间安全学院）；School of Cyberspace Science and Technology, Beijing Jiaotong University（北京交通大学网络空间安全学院）
Checkstep Research（Checkstep 研究）；University of Copenhagen（哥本哈根大学）；Qatar Computing Research Institute, HBKU（卡塔尔计算研究所，哈马德·本·哈利法大学）
The goal of stance detection is to determine the viewpoint expressed in a piece of text towards a target. These viewpoints or contexts are often expressed in many different languages depending on the user and the platform, which can be a local news outlet, a social media platform, a news forum, etc. Most research in stance detection, however, has been limited to working with a single language and on a few limited targets, with little work on cross-lingual stance detection. Moreover, non-English sources of labelled data are often scarce and present additional challenges. Recently, large multili…
Acting Flatterers via LLMs Sycophancy: Combating Clickbait with LLMs Opposing-Stance Reasoning
Yangzhou University（扬州大学）；Auburn University（亚伯拉罕大学）；University of South Dakota（南达科他大学）；Institute for Clarity in Documentation（文档清晰研究所）；Inria Paris-Rocquencourt（巴黎-罗quentourt研究所）；Rajiv Gandhi University（拉贾夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
PAST-TIDE: Prototype-Anchored Statement Tuning with Topic-Invariant Normalization for Stance Detection
Department of Computer Science and Engineering, Khulna University of Engineering & Technology（库尔纳工程技术大学计算机科学与工程系）；Department of Electrical and Computer Engineering, University of Waterloo（滑铁卢大学电气与计算机工程系）；Department of Machine Learning, Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学机器学习系）
Optimizing Social Media Annotation of HPV Vaccine Skepticism and Misinformation Using Large Language Models: An Experimental Evaluation of In-Context Learning and Fine-Tuning Stance Detection Across Multiple Models
School of Journalism and Mass Communication, University of Wisconsin–Madison（威斯康星大学麦迪逊分校新闻与大众传播学院）；Department of Electrical & Computer Engineering, University of Wisconsin–Madison（威斯康星大学麦迪逊分校电气与计算机工程系）；Department of Computer Science, University of Wisconsin–Madison（威斯康星大学麦迪逊分校计算机科学系）；Department of Psychology, University of Wisconsin–Madison（威斯康星大学麦迪逊分校心理学系）；School of Interactive Computing, Georgia Institute of Technology（佐治亚理工学院交互计算学院）；Institute of People and Technology, Georgia Institute of Technology（佐治亚理工学院人与技术研究所）
Information Sciences Institute, University of Southern California（南加州大学信息科学研究所）；University of Southern California（南加州大学）
Stance detection infers a text author's attitude towards a target. This is challenging when the model lacks background knowledge about the target. Here, we show how background knowledge from Wikipedia can help enhance the performance on stance detection. We introduce Wikipedia Stance Detection BERT (WS-BERT) that infuses the knowledge into stance encoding. Extensive results on three benchmark datasets covering social media discussions and online debates indicate that our model significantly outperforms the state-of-the-art methods on target-specific stance detection, cross-target stance detect…
