We propose and deploy an approach to continually train an instruction-following agent from feedback provided by users during collaborative interactions. During interaction, human users instruct an agent using natural language, and provide realtime binary feedback as they observe the agent following their instructions. We design a contextual bandit learning approach, converting user feedback to immediate reward. We evaluate through thousands of human-agent interactions, demonstrating 15.4% absolute improvement in instruction execution accuracy over time. We also show our approach is robust to several design variations, and that the feedback signal is roughly equivalent to the learning signal of supervised demonstration data.
East China Normal University（东华大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai AI Laboratory（上海人工智能实验室）；Xiamen University（厦门大学）；Tencent YouTu Lab（腾讯YouTu实验室）
SwitchCIT: Switching for Continual Instruction Tuning
Coordinated Science Laboratory（协调科学实验室）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Mathematics（数学系）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Continual Instruction Tuning for Large Multimodal Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；The Peng Cheng Laboratory（鹏城实验室）；Wuhan AI Research（武汉人工智能研究院）
Don't Half-listen: Capturing Key-part Information in Continual Instruction Tuning
School of Cyber Science and Engineering Nanjing University of Science and Technology China（南京理工大学信息科学与工程学院）；Meituan China（美团）；Zhipu.AI China（智谱AI）；Institute of Information Engineering Chinese Academy of Sciences China（中国科学院信息工程研究所）
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）
HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Model
School of Advanced Interdisciplinary Sciences, UCAS（UCAS先进交叉科学学院）；MAIS, CASIA（CASIA人工智能研究所）；School of Artificial Intelligence, UCAS（UCAS人工智能学院）；Centre for Artificial Intelligence and Robotics, HKISI-CAS（HKISI-CAS人工智能与机器人中心）；FKLPRIU, School of Computer and Information Engineering, Xiamen University of Technology（厦门理工学院计算机与信息工程学院）
Enhancing Multimodal Continual Instruction Tuning with BranchLoRA
Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Kyoto University（京都大学）；Tencent AI Lab（腾讯AI实验室）；Tomorrow Advancing Life（明天生命科技）
MLLM-CTBench: A Benchmark for Continual Instruction Tuning with Reasoning Process Diagnosis
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Southeast University（东南大学）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）
Learning Wisdom from Errors: Promoting LLM's Continual Relation Learning through Exploiting Error Cases
Beijing University of Posts and Telecommunications（北京邮电大学）；University of Electronic Science and Technology of China（电子科技大学）；China National Institute of Standardization（中国标准化研究院）
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Pattern Recognition Center, WeChat AI, Tencent Inc（腾讯公司微信AI模式识别中心）；School of Engineering, Westlake University（西湖大学工程学院）；Institute of Advanced Technology, Westlake Institute for Advanced Study（西湖高等研究院先进技术研究所）
Continual-NExT: A Unified Comprehension And Generation Continual Learning Framework
East China Normal University（华东师范大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Xiamen University（厦门大学）
Efficient Training of Robust Traditional Chinese LLaMA-1B on a Single Consumer GPU: Continual Pre-training, SFT, and DPO
National Tsing Hua University（台湾清华大学）；Accton Technology Corporation（智邦科技公司）；National Yang Ming Chiao Tung University（阳明交通大学）；University of Taipei（台北市立大学）
