Self-rationalizing models that also generate a free-text explanation for their predicted labels are an important tool to build trustworthy AI applications. Since generating explanations for annotated labels is a laborious and costly pro cess, recent models rely on large pretrained language models (PLMs) as their backbone and few-shot learning. In this work we explore a self-training approach leveraging both labeled and unlabeled data to further improve few-shot models, under the assumption that neither human written rationales nor annotated task labels are available at scale. We introduce a novel dual-teacher learning framework, which learns two specialized teacher models for task prediction and rationalization using self-training and distills their knowledge into a multi-tasking student model that can jointly generate the task label and rationale. Furthermore, we formulate a new loss function, Masked Label Regularization (MLR) which promotes explanations to be strongly conditioned on predicted labels. Evaluation on three public datasets demonstrate that the proposed methods are effective in modeling task labels and generating faithful rationales.
Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting
Institute of Artificial Intelligence and Future Networks, Beijing Normal University（北京师范大学人工智能与未来网络研究院）；Hong Kong Baptist University（香港浸会大学）；BNU-HKBU United International College（北京师范大学-香港浸会大学联合国际学院）；College of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
Diverge to Induce Prompting: Multi-Rationale Induction for Zero-Shot Reasoning
Department of Computer Science and Information Engineering, National Taiwan University, Taiwan（国家台湾大学计算机科学与信息工程系）；Institute of Information Science, Academia Sinica, Taiwan（学术院信息科学研究所）；AI Research Center (AINTU), National Taiwan University, Taiwan（国家台湾大学人工智能研究中心（AINTU））
Keyword-Centric Prompting for One-Shot Event Detection with Self-Generated Rationale Enhancements
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（一般人工智能国家重点实验室，智能科学与技术学院，北京大学）
Audit Before You Distill: A Controlled Workflow for Reasoning Transfer into Compact Medical Language Models
Einstein Global Advanced Technologies for Equity (E-GATE), Hospital Israelita Albert Einstein（以色列阿尔伯特·爱因斯坦医院 爱因斯坦全球公平先进技术中心（E-GATE））；Institute of Science and Technology Itaú (ICTi), Itaú Unibanco（伊塔乌联合银行 伊塔乌科学技术研究所（ICTi））；School of Sciences, São Paulo State University（圣保罗州立大学 理学院）
"The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）；Nankai University（南开大学）；The Hong Kong University of Science and Technology(Guangzhou)（香港科技大学（广州））；School of Software Engineering, State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（软件学院，人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）
Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness
LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Imperial College London（伦敦帝国学院）；Technical University of Munich（慕尼黑工业大学）；MaiNLP lab, CIS, LMU Munich（慕尼黑大学CIS研究所MaiNLP实验室）
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Alibaba Cloud Computing（阿里巴巴云计算）；Shanghai Jiao Tong University（上海交通大学）；College of Software, Zhejiang University（浙江大学软件学院）
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models
School of Engineering, Westlake University（西湖大学工程学院）；City University of Hong Kong（香港城市大学）；Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心）；Johns Hopkins University（约翰斯·霍普金斯大学）
