Demystifying Instruction Mixing for Fine-tuning Large Language Models
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；LibrAI；Monash University（莫纳什大学）；University of British Columbia（不列颠哥伦比亚大学）；The University of Melbourne（墨尔本大学）
Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. However, the procedure to optimizing the mixing of instruction datasets for LLM fine-tuning is still poorly understood. This study categorizes instructions into three primary types: NLP downstream tasks, coding, and general chat. We explore the effects of instruction tuning on different combinations of datasets on LLM performance, and find that certain instruction types are more advantageous for specific applications but can negatively impact other areas. This work provides insights into instruction mixtures, laying the foundations for future research.
Demystifying the Privacy-Utility Trade-off in LLM Interactions
Soochow University（苏州大学）；Beihang University（北京航空航天大学）；Suzhou City University（苏州城市学院）；Shanghai Key Lab of Intelligent Information Processing（上海市智能信息处理重点实验室）；Shanghai AI Lab（上海人工智能实验室）
COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning
Shenzhen Institute of Advanced Technology, CAS（中国科学院深圳先进技术研究院）；M-A-P；Institute of Automation, CAS（中国科学院自动化研究所）；Peking University（北京大学）；University of Science and Technology of China（中国科学技术大学）；HKUST（香港科技大学）；University of Manchester（曼彻斯特大学）；University of Waterloo（滑铁卢大学）；Vector Institute（向量研究所）
Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigation
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Shanghai Jiao Tong University（上海交通大学）；The University of Hong Kong（香港大学）；Hunan Artificial Intelligence and Robotics Institute Company Ltd.（湖南人工智能与机器人研究院有限公司）；Huawei Noah’s Ark Lab（华为诺亚实验室）；Peng Cheng Laboratory（鹏城实验室）
Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
University of Cambridge（剑桥大学）；OATML, University of Oxford（牛津大学OATML实验室）；University of Toronto（多伦多大学）；Shanghai Jiao Tong University（上海交通大学）；Samsung AI Center（三星人工智能中心）
No Free Lunch: Rethinking Internal Feedback for LLM Reasoning
Zhongguancun Academy（中关村学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
Brown University（布朗大学）；UC Berkeley（加州大学伯克利分校）；University of California, Los Angeles（加州大学洛杉矶分校）；HKUST（香港科技大学）；Bloomberg（彭博）；EleutherAI；Samsung R&D Institute Philippines（三星菲律宾研发中心）；Stanford University（斯坦福大学）；National University of Singapore（新加坡国立大学）；VietAI Research（VietAI研究院）；University of Potsdam（波茨坦大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）
SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models
National University of Singapore (Chongqing) Research Institute（新加坡国立大学（重庆）研究院）；Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles（重庆智能网联车辆可信感知与交互技术重点实验室）；National University of Singapore（新加坡国立大学）；Hebrew University of Jerusalem（耶路撒冷希伯来大学）；State Key Laboratory of Intelligent Vehicle Safety Technology, Chongqing, China（重庆智能车辆安全技术国家重点实验室）；CHONGQING CHANGAN AUTOMOBILE Co., Ltd（重庆长安汽车有限公司）
Integration of Large Language Models in Control of EHD Pumps for Precise Color Synthesis
Nagoya University（名古屋大学）；Chongqing University of Technology（重庆理工大学）；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）；Tokyo Institute of Technology（东京工业大学）
