Training or finetuning large-scale language models (LLMs) requires substantial computation resources, motivating recent efforts to explore parameter-efficient adaptation to downstream tasks. One approach is to treat these models as black boxes and use forward passes (Inference APIs) to interact with them. Current research focuses on adapting these black-box models to downstream tasks using gradient-free prompt optimization, but this often involves an expensive process of searching task-specific prompts. Therefore, we are motivated to study black-box language model adaptation without prompt search. Specifically, we introduce a label-enhanced cross-attention network called CrossTune, which models the semantic relatedness between the input text sequence and task-specific label descriptions. Its effectiveness is examined in the context of few-shot text classification. To improve the generalization of CrossTune, we utilize ChatGPT to generate additional training data through in-context learning. A switch mechanism is implemented to exclude low-quality ChatGPT-generated data. Through extensive experiments on seven benchmark text classification datasets, we demonstrate that our proposed approach outperforms the previous state-of-the-art gradient-free black-box tuning method by 5.7% on average. Even without using ChatGPT-augmented data, CrossTune performs better or comparably than previous black-box tuning methods, suggesting the effectiveness of our approach.
Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection
Zhejiang University（浙江大学）；School of Engineering（工程学院）；Westlake University（西湖大学）；School of Mathematics Physics and Statistics（数学、物理与统计学院）；Shanghai Polytechnic University（上海第二工业大学）；Computer Electrical and Mathematical Science and Engineering Division King Abdullah University of Science and Technology（阿卜杜拉国王科技大学计算机、电气与数学科学与工程学院）
Enhancing Black-Box Few-Shot Text Classification with Prompt-Based Data Augmentation
National University of Singapore（新加坡国立大学）；Institute of Infocomm Research (I2R), A*STAR（新加坡科技研究局资讯通信研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Kriston AI Lab（Kriston人工智能实验室）
