Recent work has described neural-network-based agents that are trained with reinforcement learning (RL) to execute language-like commands in simulated worlds, as a step towards an intelligent agent or robot that can be instructed by human users. However, the optimisation of multi-goal motor policies via deep RL from scratch requires many episodes of experience. Consequently, instruction-following with deep RL typically involves language generated from templates (by an environment simulator), which does not reflect the varied or ambiguous expressions of real users. Here, we propose a conceptually simple method for training instruction-following agents with deep RL that are robust to natural human instructions. By applying our method with a state-of-the-art pre-trained text-based language model (BERT), on tasks requiring agents to identify and position everyday objects relative to other objects in a naturalistic 3D simulated room, we demonstrate substantially-above-chance zero-shot transfer from synthetic template commands to natural instructions given by humans. Our approach is a general recipe for training any deep RL-based system to interface with human users, and bridges the gap between two research directions of notable recent success: agent-centric motor behavior and text-based representation learning.
SEIF: Self-Evolving Reinforcement Learning for Instruction Following
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University（上海数据科学 key laboratory，计算机科学与人工智能学院，复旦大学）；School of Data Science, Fudan University（数据科学学院，复旦大学）；Ant Group（蚂蚁集团）
Cornell University（康奈尔大学）；City University of New York（纽约市立大学）
We analyze language change over time in a collaborative, goal-oriented instructional task, where utility-maximizing participants form conventions and increase their expertise. Prior work studied such scenarios mostly in the context of reference games, and consistently found that language complexity is reduced along multiple dimensions, such as utterance length, as conventions are formed. In contrast, we find that, given the ability to increase instruction utility, instructors increase language complexity along these previously studied dimensions to better collaborate with increasingly skilled …
Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Kuaishou（快手）；School of Information, Renmin University of China（中国人民大学信息学院）
This paper describes an alignment-based model for interpreting natural language instructions in context. We approach instruction following as a search over plans, scoring sequences of actions conditioned on structured observations of text and the environment. By explicitly modeling both the low-level compositional structure of individual actions and the high-level structure of full plans, we are able to learn both grounded representations of sentence meaning and pragmatic constraints on interpretation. To demonstrate the model's flexibility, we apply it to a diverse set of benchmark tasks. On …
IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation
The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University（清华大学对话人工智能（CoAI）小组）；Zhipu AI（智谱AI）；University of Electronic Science and Technology of China（电子科技大学）
VCIFBench: Evaluating Complex Instruction Following for Video Understanding
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University（知识驱动人机智能工程研究中心，吉林大学）；International Center of Future Science, Jilin University（未来科学国际中心，吉林大学）
From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models
Fudan University（复旦大学）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；School of Data Science, Fudan University（复旦大学数据科学学院）；Fudan-Aishu Cognitive Intelligence Joint Research Center（复旦-爱数认知智能联合研究中心）
IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation
The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University（清华大学对话人工智能（CoAI）小组）；Zhipu AI（智谱AI）；University of Electronic Science and Technology of China（电子科技大学）；The Knowledge Engineering Group (KEG), Tsinghua University（清华大学知识工程小组（KEG））
IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languages
Nilekani Centre at AI4Bharat（AI4Bharat的Nilekani中心）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；IT University of Copenhagen（丹麦技术大学）；Microsoft（微软）
Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University（上海数据科学 key laboratory，复旦大学计算机科学与人工智能学院）；School of Data Science, Fudan University（复旦大学数据科学学院）；Ant Group（蚂蚁集团）
ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following
ByteDance China（字节跳动中国）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）
MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Following
The Pennsylvania State University（宾夕法尼亚州立大学）；The Ohio State University（俄亥俄州立大学）；Fudan University（复旦大学）；Westlake University（西湖大学）；Temple University（天普大学）
Standard architectures used in instruction following often struggle on novel compositions of subgoals (e.g. navigating to landmarks or picking up objects) observed during training. We propose a modular architecture for following natural language instructions that describe sequences of diverse subgoals. In our approach, subgoal modules each carry out natural language instructions for a specific subgoal type. A sequence of modules to execute is chosen by learning to segment the instructions and predicting a subgoal type for each segment. When compared to standard, non-modular sequence-to-sequenc…
