Although instruction tuning is widely used to adjust behavior in Large Language Models (LLMs), extensive empirical evidence and research indicates that it is primarily a process where the model fits to specific task formats, rather than acquiring new knowledge or capabilities. We propose that this limitation stems from biased features learned during instruction tuning, which differ from ideal task-specfic features, leading to learn less underlying semantics in downstream tasks. However, ideal features are unknown and incalculable, constraining past work to rely on prior knowledge to assist reasoning or training, which limits LLMs' capabilities to the developers' abilities, rather than data-driven scalable learning. In our paper, through our novel data synthesis method, DELIA (Diversity-Enhanced Learning for Instruction Adaptation), we leverage the buffering effect of extensive diverse data in LLMs training to transform biased features in instruction tuning into approximations of ideal features, without explicit prior ideal features. Experiments show DELIA's better performance compared to common instruction tuning and other baselines. It outperforms common instruction tuning by 17.07%-33.41% on Icelandic-English translation bleurt score (WMT-21 dataset, gemma-7b-it) and improves accuracy by 36.1% on formatted text generation (Llama2-7b-chat). Notably, among knowledge injection methods we've known, DELIA uniquely align the internal representations of new special tokens with their prior semantics.
Mind the Gap: Conformative Decoding to Improve Output Diversity of Instruction-Tuned Large Language Models
School of Computing University of Kent(肯特大学计算机学院);Leiden Institute of Advanced Computer Science Universiteit Leiden(莱顿先进计算机科学研究所);Cheriton School of Computer Science University of Waterloo(滑铁卢大学查里顿计算机科学学院)
Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric
College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);Institute of Modern Languages and Linguistics, Fudan University(复旦大学现代语言与语言学研究所);Institute of Trustworthy Embodied Artificial Intelligence, Fudan University(复旦大学可信具身人工智能研究所);Shanghai Collaborative Innovation Center of Intelligent Visual Computing(上海智能视觉计算协同创新中心)
CollectiveSFT: Scaling Large Language Models for Chinese Medical Benchmark with Collective Instructions in Healthcare
School of Software Engineering(软件工程学院);University of Science and Technology of China(中国科学技术大学);Shenzhen Institute of Advanced Technology(深圳先进技术研究院);Chinese Academy of Sciences(中国科学院);Xiangshui County Party School(响水县党校);School of Biomedical Engineering(生物医学工程学院);UNSW(新南威尔士大学)
Domain-Specific Improvement on Psychotherapy Chatbot Using Assistant
Czech Technical University in Prague(布拉格捷克技术大学);National Institute of Mental Health in Prague(布拉格国家精神卫生研究所);The University of Hong Kong(香港大学);Shenzhen Mental Health Centre(深圳市精神卫生中心)
LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Xiaohongshu Inc(小红书公司);Tsinghua University(清华大学)
ChemOrch: Empowering LLMs with Chemical Intelligence via Synthetic Instructions
Department of Computer Science and Engineering, University of Notre Dame(诺丁汉大学计算机科学与工程系);MIT(麻省理工学院);CalTech(加州理工学院);MBZUAI(穆斯林人工智能研究所);CMU(卡内基梅隆大学);Department of Chemistry & Biochemistry, University of Notre Dame(诺丁汉大学化学与生物化学系)
Self-Instruct: Aligning Language Models with Self-Generated Instructions
University of Washington(华盛顿大学);Tehran Polytechnic(德黑兰理工大学);Arizona State University(亚利桑那州立大学);Johns Hopkins University(约翰斯·霍普金斯大学);Allen Institute for AI(艾伦人工智能研究所)
Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs
National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(国家多媒体信息处理重点实验室,计算机学院,北京大学);ByteDance(字节跳动)