Recently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Instruction-tuned models have significantly outperformed multitask learning models (without instruction); however they are far from state-of-the-art task-specific models. Conventional approaches to improve model performance via creating datasets with large number of task instances or architectural changes in the model may not be feasible for non-expert users. However, they can write alternate instructions to represent an instruction task. Is Instruction-augmentation helpful? We augment a subset of tasks in the expanded version of NATURAL INSTRUCTIONS with additional instructions and find that it significantly improves model performance (up to 35%), especially in the low-data regime. Our results indicate that an additional instruction can be equivalent to ~200 data samples on average across tasks.
What Makes Instruction Learning Hard? An Investigation and a New Challenge in a Synthetic Environment
Allen Institute for AI(艾伦人工智能研究所)
指令学习范式--模型仅从任务描述中学习执行新任务--在通用模型研究中变得流行。然而,大型Transformer模型作为指令学习者的能力仍然知之甚少。我们使用受控的合成环境来表征这种能力。具体来说,我们使用决定给定字符串是否匹配正则表达式(视为指令)的任务来识别使指令学习具有挑战性的任务、指令和实例的属性。例如,我们发现我们的模型,一个微调的基于T5的text 2 text Transformer,与大型正则语言斗争,这表明不太精确的指令对模型来说是一个挑战。另外,需要跟踪先前步骤的较长上下文的指令执行也更困难。我们使用我们的研究结果系统地构建了一个具有挑战性的指令学习数据集,我们称之为硬RegSet。通过对硬RegSet进行微调,我们的大型Transformer只能正确解释65.6%的测试指令(至少90%的准确率),以及分布外泛化设置中11%-24%的指令。我们提出了硬RegSet作为一个具有挑战性的指令学习任务,和一个可控的环境研究指令学习。
Enhancing Speech Instruction Understanding and Disambiguation in Robotics via Speech Prosody
IT University of Copenhagen(丹麦技术大学);University of Florida(佛罗里达大学);Brown University(布朗大学);Intelligent Agents Research Group(智能代理研究组);Intelligent Robot Lab(智能机器人实验室)
InstructIE: A Bilingual Instruction-based Information Extraction Dataset
Zhejiang University(浙江大学);Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph(浙江大学-蚂蚁集团知识图谱联合实验室);Ant Group(蚂蚁集团);University of Edinburgh(爱丁堡大学)
GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning
Peking University(北京大学);Mila(米拉研究所);The University of Hong Kong(香港大学);University of Oxford(牛津大学);Hong Kong University of Science and Technology(香港科技大学)
Instruction-Grounded Visual Projectors for Continual Learning of Generative Vision-Language Models
School of Computer Science and Engineering, Chung-Ang University(Chung-Ang 大学计算机科学与工程学院);School of Computer Science, University of Birmingham(布拉德福德大学计算机科学学院)
Learning to Follow Instructions in Text-Based Games
University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(矢量人工智能研究所);Schwartz Reisman Institute for Technology and Society(施瓦茨·赖斯曼技术与社会研究所)
Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection
Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology(哈尔滨工业大学社会计算与信息检索研究中心);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Du Xiaoman Technology (Beijing) Co., Ltd.(度小满科技(北京)有限公司)