Recent works have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. However, creating such datasets is difficult and most works rely on manual curation or proprietary language models. Automatic data curation is difficult as it is still not clear how we can define diversity for instruction tuning, how diversity and quality depend on one other, and how we can optimize dataset quality and diversity. To resolve these issue, we propose a new algorithm, Quality-Diversity Instruction Tuning (QDIT). QDIT provides a simple method to simultaneously control dataset diversity and quality, allowing us to conduct an in-depth study on the effect of diversity and quality on instruction tuning performance. From this study we draw two key insights (1) there is a natural tradeoff between data diversity and quality and (2) increasing data diversity significantly improves the worst case instruction following performance, therefore improving robustness. We validate the performance of QDIT on several large scale instruction tuning datasets, where we find it can substantially improve worst and average case performance compared to quality-driven data selection.
Mind the Gap: Conformative Decoding to Improve Output Diversity of Instruction-Tuned Large Language Models
School of Computing University of Kent（肯特大学计算机学院）；Leiden Institute of Advanced Computer Science Universiteit Leiden（莱顿先进计算机科学研究所）；Cheriton School of Computer Science University of Waterloo（滑铁卢大学查里顿计算机科学学院）
Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；Institute of Modern Languages and Linguistics, Fudan University（复旦大学现代语言与语言学研究所）；Institute of Trustworthy Embodied Artificial Intelligence, Fudan University（复旦大学可信具身人工智能研究所）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）
LongForm: Effective Instruction Tuning with Reverse Instructions
Center for Information and Language Processing, LMU Munich（慕尼黑大学信息与语言处理中心）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Language Technology Lab, University of Cambridge（剑桥大学语言技术实验室）
On the Effect of Instruction Tuning Loss on Generalization
Dept. of Electrical Engineering（电气工程系）；Indian Institute of Technology Delhi（印度理工学院德里）；Media and Data Science Research（媒体与数据科学研究）；Adobe Inc., India（Adobe公司，印度）
Instruction tuning enables pretrained language models to perform new tasks from inference-time natural language descriptions. These approaches rely on vast amounts of human supervision in the form of crowdsourced datasets or user interactions. In this work, we introduce Unnatural Instructions: a large dataset of creative and diverse instructions, collected with virtually no human labor. We collect 64,000 examples by prompting a language model with three seed examples of instructions and eliciting a fourth. This set is then expanded by prompting the model to rephrase each instruction, creating …
We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022 (Chung et al., 2022). Through careful ablation studies on the Flan Collection of tasks and methods, we tease apart the effect of design decisions which enable Flan-T5 to outperform prior work by 3-17%+ across evaluation settings. We find task balancing and enrichment techniques are overlooked but critical to effective instruction tuning, and in particular, training with mixed prompt settings (zero-shot, few-shot, and chain-of-thought) actually yields stronger (2%+) perfor…
University of Southern California（南加州大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；The Pennsylvania State University（宾夕法尼亚州立大学）；University of California, Davis（加利福尼亚大学戴维斯分校）
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models
Department of Computer Science, School of Computing, Institute of Science Tokyo（计算机科学系、计算学院、科学东京研究所）；National Institute of Advanced Industrial Science and Technology（国家先进工业科学与技术研究院）；Research and Development Center for Large Language Models, NII（大型语言模型研究与开发中心、日本科学振兴会）
A Survey on Data Selection for LLM Instruction Tuning
Harbin Institute of Technology（哈尔滨工业大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Harbin Institute of Technology (Weihai)（哈尔滨工业大学（威海））
Instruction tuning is an emergent paradigm in NLP wherein natural language instructions are leveraged with language models to induce zero-shot performance on unseen tasks. Instructions have been shown to enable good performance on unseen tasks and datasets in both large and small language models. Dialogue is an especially interesting area to explore instruction tuning because dialogue systems perform multiple kinds of tasks related to language (e.g., natural language understanding and generation, domain-specific interaction), yet instruction tuning has not been systematically explored for dial…
MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
Washington University in St. Louis（华盛顿大学圣路易斯分校）；The University of British Columbia（不列颠哥伦比亚大学）；Google Research（谷歌研究）；Virginia Tech（弗吉尼亚理工大学）；University of California, Davis（加州大学戴维斯分校）；Sony AI（索尼人工智能）；University of California, Berkeley（加州大学伯克利分校）
Synthesizing Instruction-Tuning Datasets with Contrastive Decoding
Department of Computer Science, School of Computing, Institute of Science Tokyo（东京科学大学计算机科学系）；National Institute of Advanced Industrial Science and Technology（国家先进工业科学与技术研究院）；Research and Development Center for Large Language Models, NII（大型语言模型研究与开发中心，日本信息产业技术综合研究所）
Demystifying Instruction Mixing for Fine-tuning Large Language Models
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；LibrAI；Monash University（莫纳什大学）；University of British Columbia（不列颠哥伦比亚大学）；The University of Melbourne（墨尔本大学）
