Interactive robot learning is a challenging problem as the robot is present with human users who expect the robot to learn novel skills to solve novel tasks perpetually with sample efficiency. In this work we present a framework for robots to continually learn tasks and visuo-motor skills and query for novel skills via dialog interactions with human users. Our robot agent maintains a skill library, and uses an existing LLM to perform grounded dialog interactions to query unknown skills from real human users. We developed a novel visual-motor control policy Action Chunking Transformer with Low Rank Adaptation (ACT-LoRA) that can continually learn novel skills using only a few demonstrations which is critical in human-robot interaction scenarios. The paper has twin goals: Firstly to demonstrate better continual learning in simulation; and secondly, to demonstrate the use of our dialog based learning framework in a realistic human-robot interaction use case. Our ACT-LoRA policy consistently outperforms a GMM-LoRA baseline on multiple continual learning simulation benchmarks by achieving > 300% improvements on novel skills, while achieving comparable performance in existing skills. Moreover, with our IRB approved human-subjects study we demonstrate that our dialog based continual learning framework allows users to teach robots cooking skills successfully (100%) while spending a higher ratio of time on finishing an auxiliary distraction tasks in the test phase of the study compared to a non-learning language based agent (p < 0.001).
Center for Artificial Intelligence Research (CAiRE)（人工智能研究中心（CAiRE））；The Hong Kong University of Science and Technology（香港科技大学）
Considerable progress has been made towards conversational models that generate coherent and fluent responses by training large language models on large dialogue datasets. These models have little or no control of the generated responses and miss two important features: continuous dialogue skills integration and seamlessly leveraging diverse knowledge sources. In this paper, we propose the Adapter-Bot, a dialogue model that uses a fixed backbone conversational model such as DialGPT (Zhang et al., 2019) and triggers on-demand dialogue skills (e.g., emphatic response, weather information, movie …
Tsinghua University（清华大学）；Beijing Innovation Center for Future Chip（北京未来芯片创新中心）
Imitation learning (IL) algorithms have shown promising results for robots to learn skills from expert demonstrations. However, they need multi-task demonstrations to be provided at once for acquiring diverse skills, which is difficult in real world. In this work we study how to realize continual imitation learning ability that empowers robots to continually learn new tasks one by one, thus reducing the burden of multi-task IL and accelerating the process of new task learning at the same time. We propose a novel trajectory generation model that employs both a generative adversarial network and…
University of Hamburg（汉堡大学）；SoftBank Robotics Europe（软银机器人欧洲公司）
Service robots need to show appropriate social behaviour in order to be deployed in social environments such as healthcare, education, retail, etc. Some of the main capabilities that robots should have are navigation and conversational skills. If the person is impatient, the person might want a robot to navigate faster and vice versa. Linguistic features that indicate politeness can provide social cues about a person's patient and impatient behaviour. The novelty presented in this paper is to dynamically incorporate politeness in robotic dialogue systems for navigation. Understanding the polit…
Never-Ending Behavior-Cloning Agent for Robotic Manipulation
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（机器人国家重点实验室，沈阳自动化研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；School of Automation Science and Engineering, South China University of Technology（自动化科学与工程学院，华南理工大学）；Mohamed bin Zayed University of Artificial Intelligence（马斯达尔大学人工智能学院）
Knowledge Fusion of Large Language Models Via Modular SkillPacks
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Hong Kong Polytechnic University（香港理工大学）；Nanyang Technological University（南洋理工大学）；Huazhong University of Science and Technology（华中科技大学）
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Peking University（北京大学）；Google Research（谷歌研究院）
Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications. Adapting or posttraining an LM using an unlabeled domain corpus can produce even better performance for end-tasks in the domain. This paper proposes the problem of continually extending an LM by incrementally post-train the LM with a sequence of unlabeled domain corpora to expand its knowledge without forgetting its previous skills. The goal is to improve the few-shot end-task learning in these domains. The resulting system is called CPT (Continual PostTraining), which to our knowledge…
The largest store of continually updating knowledge on our planet can be accessed via internet search. In this work we study giving access to this information to conversational agents. Large language models, even though they store an impressive amount of knowledge within their weights, are known to hallucinate facts when generating dialogue (Shuster et al., 2021); moreover, those facts are frozen in time at the point of model training. In contrast, we propose an approach that learns to generate an internet search query based on the context, and then conditions on the search results to finally …
Don't Wait to Reply: Towards Responsive yet Thoughtful Dialogue through Proactive Thinking
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University（清华大学计算机科学与技术系、人工智能研究院）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Tencent（腾讯）
Recent work on large language models relies on the intuition that most natural language processing tasks can be described via natural language instructions. Language models trained on these instructions show strong zero-shot performance on several standard datasets. However, these models even though impressive still perform poorly on a wide range of tasks outside of their respective training and evaluation sets. To address this limitation, we argue that a model should be able to keep extending its knowledge and abilities, without forgetting previous skills. In spite of the limited success of C…
Large Language Models (LMs) are known to encode world knowledge in their parameters as they pretrain on a vast amount of web corpus, which is often utilized for performing knowledge-dependent downstream tasks such as question answering, fact-checking, and open dialogue. In real-world scenarios, the world knowledge stored in the LMs can quickly become outdated as the world changes, but it is non-trivial to avoid catastrophic forgetting and reliably acquire new knowledge while preserving invariant knowledge. To push the community towards better maintenance of ever-changing LMs, we formulate a ne…
University of Southern California（南加州大学）；Amazon（亚马逊）
In order to be effective general purpose machines in real world environments, robots not only will need to adapt their existing manipulation skills to new circumstances, they will need to acquire entirely new skills on-the-fly. A great promise of continual learning is to endow robots with this ability, by using their accumulated knowledge and experience from prior skills. We take a fresh look at this problem, by considering a setting in which the robot is limited to storing that knowledge and experience only in the form of learned skill policies. We show that storing skill policies, careful pr…
SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）
OrthoSkillVLA: Continual Skill Learning via Gradient-Informed Skill Subspace Adaptation
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Electronic Science and Engineering, Southeast University（东南大学电子科学与工程学院）
Extreme-RGMT: Continual Learning of Highly Dynamic Skills for Robust Generalist Humanoid Control
School of Automation, Beijing Institute of Technology（北京理工大学自动化学院）；Humanoid Robotics (Shanghai) Co., Ltd.（人形机器人（上海）有限公司）；School of Mechanical Engineering, Shandong University（山东大学机械工程学院）
Growable and Interpretable Neural Control with Online Continual Learning for Autonomous Lifelong Locomotion Learning Machines
Embodied AI and Neurorobotics Laboratory, SDU Biorobotics, The Mærsk Mc-Kinney Møller Institute, The University of Southern Denmark, Odense, Denmark（具身体验人工智能与神经机器人实验室，SDU生物机器人学，马士克麦金尼·莫勒研究所，南丹麦大学，丹麦欧登塞）；Bio-Inspired Robotics and Neural Engineering Laboratory, School of Information Science and Technology, Vidyasirimedhi Institute of Science and Technology, Rayong, Thailand（生物启发机器人与神经工程实验室，信息科学与技术学院，维达西里梅迪科技研究所，泰国拉耶昂）
