Accepted at AAAI 2022; 7 pages main, 28 pages total, 29 figures; Version 3 uses a new test set for EDH instances that restrict evaluation to state changes only on task-relevant objects
Robots operating in human spaces must be able to engage in natural language interaction with people, both understanding and executing instructions, and using conversation to resolve ambiguity and recover from mistakes. To study this, we introduce TEACh, a dataset of over 3,000 human--human, interactive dialogues to complete household tasks in simulation. A Commander with access to oracle information about a task communicates in natural language with a Follower. The Follower navigates through and interacts with the environment to complete tasks varying in complexity from "Make Coffee" to "Prepare Breakfast", asking questions and getting additional information from the Commander. We propose three benchmarks using TEACh to study embodied intelligence challenges, and we evaluate initial models' abilities in dialogue understanding, language grounding, and task execution.
Your Robot Will Feel You Now: Empathy in Robots and Embodied Agents
School of Computing Science, Simon Fraser University（计算科学学院，西蒙弗雷泽大学）；School of Interactive Arts and Technology, Simon Fraser University（交互艺术与技术学院，西蒙弗雷泽大学）
MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Science（中国科学院大学人工智能学院）；Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Tongji University（同济大学）
LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Robotics
Automation and Control Institute, Faculty of Electrical Engineering, TU Wien（自动化与控制研究所，电气工程学院，维也纳技术大学）；AIT Austrian Institute of Technology GmbH, Center for Vision, Automation and Control（奥地利技术研究所（AIT）有限公司，视觉、自动化与控制中心）
March in Chat: Interactive Prompting for Remote Embodied Referring Expression
Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
