Instruction tuning plays a pivotal role in Code Large Language Models (Code LLMs) for the task of program synthesis. Presently, two dominant paradigms for collecting tuning data are natural-instruct (human-written) and self-instruct (automatically generated). Natural-instruct includes diverse and correct codes but lacks instruction-code pairs, and exists improper code formats like nested single-line codes. In contrast, self-instruct automatically generates proper paired data. However, it suffers from low diversity due to generating duplicates and cannot ensure the correctness of codes. To bridge the both paradigms, we propose \textbf{Semi-Instruct}. It first converts diverse but improper codes from natural-instruct into proper instruction-code pairs through a method similar to self-instruct. To verify the correctness of generated codes, we design a novel way to construct test cases by generating cases' inputs and executing correct codes from natural-instruct to get outputs. Finally, diverse and correct instruction-code pairs are retained for instruction tuning. Experiments show that semi-instruct is significantly better than natural-instruct and self-instruct. Furthermore, the performance steadily improves as data scale increases.
Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models
The Chinese University of Hong Kong(香港中文大学);University of International Relations(国际关系大学);Tencent Jarvis Lab(腾讯Jarvis实验室);Westlake University(西湖大学);King’s College London(伦敦国王学院)
MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistants
South China Agricultural University(华南农业大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of Electronic Science and Technology of China(电子科技大学)
Tower+: Bridging Generality and Translation Specialization in Multilingual LLMs
Unbabel;Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico & Universidade de Lisboa (Lisbon ELLIS Unit)(里斯本大学技术学院及里斯本大学(里斯本ELLIS单位));MICS, CentraleSupélec, Université Paris-Saclay(MICS,CentraleSupélec,巴黎萨克雷大学)
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Bytedance Inc.(字节跳动公司)
Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室)
ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset
School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai, China(上海交通大学航空航天学院);Shanghai Innovation Institute, Shanghai, China(上海创新研究院);School of Data Science, Fudan University, China(复旦大学数据科学学院)
InstructProtein: Aligning Human and Protein Language via Knowledge Instruction
Zhejiang University(浙江大学);ZJU-Hangzhou Global Scientific and Technological Innovation Center(浙江大学杭州国际科创中心);AZFT Joint Lab for Knowledge Engine(AZFT知识引擎联合实验室)
MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response
University of Pennsylvania(宾夕法尼亚大学);Queen Mary University of London(伦敦大学玛丽皇后学院);Carnegie Mellon University(卡内基梅隆大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Waterloo(滑铁卢大学)
MusiLingo 通过单一投影层将冻结音乐音频模型 MERT 与冻结 LLM 对齐,并构建 MusicInstruct 数据集,实现音乐描述生成与音乐查询响应的性能提升。
Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compression
Faculty of Electronic and Information Engineering, Xi’an Jiaotong University(西安交通大学电子与信息工程学院);Queen Mary University of London(伦敦大学玛丽女王学院);City University of Hong Kong(香港城市大学)
ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Tencent Hunyuan(腾讯混元);Shenzhen Loop Area Institute(深圳河套学院);Amphion Technology Co., Ltd.(Amphion科技有限公司);Tsinghua University(清华大学)
Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);Bosch Corporate Research(博世企业研究);King Abdullah University of Science and Technology(卡布斯大学)