Recent years have witnessed increasing interests in prompt-based learning in which models can be trained on only a few annotated instances, making them suitable in low-resource settings. When using prompt-based learning for text classification, the goal is to use a pre-trained language model (PLM) to predict a missing token in a pre-defined template given an input text, which can be mapped to a class label. However, PLMs built on the transformer architecture tend to generate similar output embeddings, making it difficult to discriminate between different class labels. The problem is further exacerbated when dealing with classification tasks involving many fine-grained class labels. In this work, we alleviate this information diffusion issue, i.e., different tokens share a large proportion of similar information after going through stacked multiple self-attention layers in a transformer, by proposing a calibration method built on feature transformations through rotation and scaling to map a PLM-encoded embedding into a new metric space to guarantee the distinguishability of the resulting embeddings. Furthermore, we take the advantage of hyperbolic embeddings to capture the hierarchical relations among fine-grained class-associated token embedding by a coarse-to-fine metric learning strategy to enhance the distinguishability of the learned output embeddings. Extensive experiments on the three datasets under various settings demonstrate the effectiveness of our approach. Our code can be found at https://github.com/donttal/TARA.
Surprise Calibration for Better In-Context Learning
School of Information Management, Wuhan University（武汉大学信息管理学院）；Center for the Studies of Information Resources, Wuhan University（武汉大学信息资源研究中心）；School of Economics and Management, Nanjing University of Science and Technology（南京理工大学经济管理学院）
N2C2: Nearest Neighbor Enhanced Confidence Calibration for Cross-Lingual In-Context Learning
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Meituan（美团）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
$k$NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference
University of Science and Technology of China（中国科学技术大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Baidu Inc.（百度公司）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
Large Language Models are Miscalibrated In-Context Learners
Language Technology Lab, University of Cambridge（剑桥大学语言技术实验室）；Center for Artificial Intelligence and Data Science, University of Würzburg（乌尔姆大学人工智能与数据科学中心）
SGIC: A Self-Guided Iterative Calibration Framework for RAG
NLP 2 CT Lab, Department of Computer and Information Science, University of Macau（自然语言处理2CT实验室，计算机与信息科学系，澳门大学）；Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China（计算与智能学院，哈尔滨工业大学深圳校区，中国）
