The effectiveness of contrastive learning technology in natural language processing tasks is yet to be explored and analyzed. How to construct positive and negative samples correctly and reasonably is the core challenge of contrastive learning. It is even harder to discover contrastive objects in multi-label text classification tasks. There are very few contrastive losses proposed previously. In this paper, we investigate the problem from a different angle by proposing five novel contrastive losses for multi-label text classification tasks. These are Strict Contrastive Loss (SCL), Intra-label Contrastive Loss (ICL), Jaccard Similarity Contrastive Loss (JSCL), Jaccard Similarity Probability Contrastive Loss (JSPCL), and Stepwise Label Contrastive Loss (SLCL). We explore the effectiveness of contrastive learning for multi-label text classification tasks by the employment of these novel losses and provide a set of baseline models for deploying contrastive learning techniques on specific tasks. We further perform an interpretable analysis of our approach to show how different components of contrastive learning losses play their roles. The experimental results show that our proposed contrastive losses can bring improvement to multi-label text classification tasks. Our work also explores how contrastive learning should be adapted for multi-label text classification tasks.
Mao-Zedong At SemEval-2023 Task 4: Label Represention Multi-Head Attention Model With Contrastive Learning-Enhanced Nearest Neighbor Mechanism For Multi-Label Text Classification
Qifu Technology（奇富科技）；Peking University（北京大学）
本文针对 SemEval-2023 任务 4 的多标签人类价值观识别问题，提出基于 Roberta 的标签表示多头注意力模型，并结合对比学习增强的 K 近邻机制，最终在测试集上取得 F1 分数 0.533、排名第四的成绩。
Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation
SKLSDE, School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机学院软件开发环境国家重点实验室）；Beijing Advanced Institution on Big Data and Brain Computing, Beihang University（北京航空航天大学大数据与脑机智能高精尖创新中心）；School of Electrical Engineering and Computer Science, University of Ottawa（渥太华大学电气工程与计算机科学学院）
Contrastive Explanations for Model Interpretability
Bar Ilan University（巴伊兰大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Paul G. Allen School of Computer Science and Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）
Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discovery
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件学院）；City University of Hong Kong（香港城市大学）；Singapore Management University（新加坡管理大学）；School of Information and Communication Engineering, Xi’an Jiaotong University（西安交通大学信息与通信工程学院）
HC$^2$L: Hybrid and Cooperative Contrastive Learning for Cross-lingual Spoken Language Understanding
University of Science and Technology Beijing（北京科技大学）；Agency for Science, Technology and Research（新加坡科技研究局）；Nanyang Technological University（南洋理工大学）；University of Technology Sydney（悉尼科技大学）
