We study instruction following in multi-task reinforcement learning, where an agent must zero-shot execute novel tasks not seen during training. In this setting, linear temporal logic (LTL) has recently been adopted as a powerful framework for specifying structured, temporally extended tasks. While existing approaches successfully train generalist policies, they often struggle to effectively capture the rich logical and temporal structure inherent in LTL specifications. In this work, we address these concerns with a novel approach to learn structured task representations that facilitate training and generalisation. Our method conditions the policy on sequences of Boolean formulae constructed from a finite automaton of the task. We propose a hierarchical neural architecture to encode the logical structure of these formulae, and introduce an attention mechanism that enables the policy to reason about future subgoals. Experiments in a variety of complex environments demonstrate the strong generalisation capabilities and superior performance of our approach.
LTL2Action: Generalizing LTL Instructions for Multi-Task RL
University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(向量人工智能研究所);Schwartz Reisman Institute for Technology and Society(施瓦茨·赖斯曼技术与社会研究所)
Learning to Follow Instructions in Text-Based Games
University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(矢量人工智能研究所);Schwartz Reisman Institute for Technology and Society(施瓦茨·赖斯曼技术与社会研究所)
GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning
Peking University(北京大学);Mila(米拉研究所);The University of Hong Kong(香港大学);University of Oxford(牛津大学);Hong Kong University of Science and Technology(香港科技大学)
ConformalNL2LTL: Translating Natural Language Instructions into Temporal Logic Formulas with Conformal Correctness Guarantees
Department of Electrical and Systems Engineering, Washington University in St Louis(华盛顿大学圣路易斯分校电子与系统工程系);Department of Computer Science, University of Southern California(南加州大学计算机科学系)
DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition
Seltasquare Seoul, Korea(首尔韩国Seltasquare);Seoul National University College of Medicine(首尔国立大学医学院);Biomedical Research Institute, Seoul National University Hospital(首尔国立大学医院生物医学研究所)
A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks
LIA, Avignon Université(阿维尼翁大学 LIA 实验室);Zenidoc(Zenidoc 公司);Nantes Université(南特大学);École Centrale Nantes(南特中央理工学院);CNRS(法国国家科学研究中心);LS2N, UMR 6004(南特数字科学实验室)
From One-Shot Generation to Incremental Music Composition: Adapting a General-Purpose Instruction LLM for Persistent Symbolic Editing
Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio)(里约热内卢天主教大学);Sorbonne Université(索邦大学);LIP6(巴黎第六大学计算机科学实验室);Universidade Federal do Rio de Janeiro (UFRJ)(里约热内卢联邦大学)
M$^2$PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning
Rochester Institute of Technology(罗切斯特理工学院);U.S. Naval Research Laboratory(美国海军研究实验室);Harvard Medical School(哈佛医学院);ByteDance(字节跳动);Meta AI;Meituan(美团);University of Science and Technology of China(中国科学技术大学);Shanghai Academy of AI for Science(上海人工智能科学研究院);Fudan University(复旦大学);University of Missouri - Kansas City(密苏里大学堪萨斯城分校);University of California - Davis(加利福尼亚大学戴维斯分校)
LLMs-based Few-Shot Disease Predictions using EHR: A Novel Approach Combining Predictive Agent Reasoning and Critical Agent Instruction
Emory University(埃默里大学);University of Washington(华盛顿大学);UCSD(加利福尼亚大学圣迭戈分校);Rollins School of Public Health, Emory University(埃默里大学罗林斯公共卫生学院);School of Medicine, Emory University(埃默里大学医学院)
Are Large Language Models Table-based Fact-Checkers?
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance
Indiana University Bloomington(印第安纳大学伯明顿分校);Center for Complex Networks and Systems(复杂网络与系统中心)