Instruction-tuning can be substantially optimized through enhanced diversity, resulting in models capable of handling a broader spectrum of tasks. However, existing data employed for such tuning often exhibit an inadequate coverage of individual domains, limiting the scope for nuanced comprehension and interactions within these areas. To address this deficiency, we propose Explore-Instruct, a novel approach to enhance the data coverage to be used in domain-specific instruction-tuning through active exploration via Large Language Models (LLMs). Built upon representative domain use cases, Explore-Instruct explores a multitude of variations or possibilities by implementing a search algorithm to obtain diversified and domain-focused instruction-tuning data. Our data-centric analysis validates the effectiveness of this proposed approach in improving domain-specific instruction coverage. Moreover, our model's performance demonstrates considerable advancements over multiple baselines, including those utilizing domain-specific data enhancement. Our findings offer a promising opportunity to improve instruction coverage, especially in domain-specific contexts, thereby advancing the development of adaptable language models. Our code, model weights, and data are public at \url{https://github.com/fanqiwan/Explore-Instruct}.
TAG-INSTRUCT: Controlled Instruction Complexity Enhancement through Structure-based Augmentation
Peking University(北京大学);Southern University of Science and Technology(南方科技大学);Tongji University(同济大学);Shanghai University of Finance and Economics(上海财经大学);The University of Hong Kong(香港大学)
MoDS: Model-oriented Data Selection for Instruction Tuning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Wuhan AI Research(武汉人工智能研究院)
RAISE: Reinforced Adaptive Instruction Selection For Large Language Models
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Peng Cheng Laboratory(鹏城实验室);University of Illinois Chicago(伊利诺伊大学香槟分校);Seoul National University(首尔国立大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳))
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models
Department of Computer Science, School of Computing, Institute of Science Tokyo(计算机科学系、计算学院、科学东京研究所);National Institute of Advanced Industrial Science and Technology(国家先进工业科学与技术研究院);Research and Development Center for Large Language Models, NII(大型语言模型研究与开发中心、日本科学振兴会)
What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Information, Renmin University of China(中国人民大学信息学院);Meituan Group(美团集团)
X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructions
State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统国家重点实验室);Institute of Automation, CAS(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Following
The Pennsylvania State University(宾夕法尼亚州立大学);The Ohio State University(俄亥俄州立大学);Fudan University(复旦大学);Westlake University(西湖大学);Temple University(天普大学)
AFD-INSTRUCTION: A Comprehensive Antibody Instruction Dataset with Functional Annotations for LLM-Based Understanding and Design
National Institute for Data Science in Health and Medicine(健康医学数据科学国家研究院);Institute of Artificial Intelligence(人工智能研究院);School of Life Sciences(生命科学学院);School of Informatics(信息学院);State Key Laboratory of Vaccines for Infectious Diseases(传染病疫苗国家重点实验室);Xiang An Biomedicine Laboratory(翔安生物医学实验室)
ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following
ByteDance China(字节跳动中国);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室)