Language model fine-tuning is essential for modern natural language processing, but is computationally expensive and time-consuming. Further, the effectiveness of fine-tuning is limited by the inclusion of training examples that negatively affect performance. Here we present a general fine-tuning method that we call information gain filtration for improving the overall training efficiency and final performance of language model fine-tuning. We define the information gain of an example as the improvement on a test metric after training on that example. A secondary learner is then trained to approximate this quantity. During fine-tuning, this learner selects informative examples and skips uninformative ones. We show that our method has consistent improvement across datasets, fine-tuning tasks, and language model architectures. For example, we achieve a median perplexity of 54.0 on a books dataset compared to 57.3 for standard fine-tuning. We present statistical evidence that offers insight into the improvements of our method over standard fine-tuning. The generality of our method leads us to propose a new paradigm for language model fine-tuning -- we encourage researchers to release pretrained secondary learners on common corpora to promote efficient and effective fine-tuning, thereby improving the performance and reducing the overall energy footprint of language model fine-tuning.
Selecting What Matters: Semantic Compression-Guided Selective Pooling for Long-Context Embeddings
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；National University of Singapore（新加坡国立大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
University of Washington（华盛顿大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Stanford University（斯坦福大学）；Georgia Institute of Technology（佐治亚理工学院）；University of California, Riverside（加州大学河滨分校）；Microsoft Research NYC（微软研究院纽约分部）
Training Language Models with Language Feedback at Scale
New York University（纽约大学）；FAR AI；University of the Basque Country UPV/EHU（巴斯克大学）；University of Sussex（萨塞克斯大学）；Genentech（基因泰克公司）；CIFAR LMB（加拿大高级研究院分子生物学与遗传学实验室）；Anthropic（Anthropic公司）
