With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs). Previous research mainly focuses on selecting individual high-quality instructions. However, these works overlooked the joint interactions and dependencies between different categories of instructions, leading to suboptimal selection strategies. Moreover, the nature of these interaction patterns remains largely unexplored, let alone optimize the instruction set with regard to them. To fill these gaps, in this paper, we: (1) systemically investigate interaction and dependency patterns between different categories of instructions, (2) manage to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning. Experimental results across different LLMs demonstrate improved performance over strong baselines on widely adopted benchmarks.
Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University（上海数据科学 key laboratory，复旦大学计算机科学与人工智能学院）；School of Data Science, Fudan University（复旦大学数据科学学院）；Ant Group（蚂蚁集团）
Benchmarking Complex Instruction-Following with Multiple Constraints Composition
Tsinghua University（清华大学）；University of Electronic Science and Technology of China（电子科技大学）；Zhipu AI（智谱AI）；China University of Geosciences（中国地质大学）；Central China Normal University（华中师范大学）
GATEAU: Selecting Influential Samples for Long Context Alignment
Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Peking University（北京大学）；DeepLang AI；Institute for AI, Tsinghua University（清华大学人工智能研究院）
Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
AppBench: Planning of Multiple APIs from Various APPs for Complex User Instruction
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）；Beihang University（北京航空航天大学）；The University of Edinburgh（爱丁堡大学）
DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planning
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences (CASIA)（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院（CASIA））；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，北京大学计算机科学学院）；Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）；OpenGVLab, Shanghai AI Laboratory（上海人工智能实验室，OpenGVLab）
ComposerX: Multi-Agent Symbolic Music Composition with LLMs
University of Rochester（罗切斯特大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Hong Kong University of Science and Technology（香港科技大学）；University of Surrey（萨里大学）；The University of Manchester（曼彻斯特大学）；Queen Mary University of London（伦敦玛丽女王大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
GDLLM: A Global Distance-aware Modeling Approach Based on Large Language Models for Event Temporal Relation Extraction
School of Computer Science and Technology, Dalian University of Technology, China（大连理工大学计算机科学与技术学院）；Luddy School of Informatics, Computing, and Engineering, Indiana University Indianapolis, the United States（印第安纳大学印第安纳波利斯分校信息学院）
Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions
Department of Computer Science & Information Systems BITS Pilani（比拉理工学院皮拉尼校区计算机科学与信息系统系）；School of Computer Engineering, KIIT Bhubaneshwar（KIIT布巴内斯瓦尔分校计算机工程学院）
