Domestic and service robots have the potential to transform industries such as health care and small-scale manufacturing, as well as the homes in which we live. However, due to the overwhelming variety of tasks these robots will be expected to complete, providing generic out-of-the-box solutions that meet the needs of every possible user is clearly intractable. To address this problem, robots must therefore not only be capable of learning how to complete novel tasks at run-time, but the solutions to these tasks must also be informed by the needs of the user. In this paper we demonstrate how behaviour trees, a well established control architecture in the fields of gaming and robotics, can be used in conjunction with natural language instruction to provide a robust and modular control architecture for instructing autonomous agents to learn and perform novel complex tasks. We also show how behaviour trees generated using our approach can be generalised to novel scenarios, and can be re-used in future learning episodes to create increasingly complex behaviours. We validate this work against an existing corpus of natural language instructions, demonstrate the application of our approach on both a simulated robot solving a toy problem, as well as two distinct real-world robot platforms which, respectively, complete a block sorting scenario, and a patrol scenario.
Agentic AI for Gravitational Wave Data Analysis: A Head-to-Head Comparison of Coding Agents Executing a Matched Filter Pipeline on Einstein Telescope Simulated Data
City Editing: Hierarchical Agentic Execution for Dependency-Aware Urban Geospatial Modification
University of Kansas, Lawrence, KS, USA（堪萨斯大学）；Seoul National University, Seoul, South Korea（首尔国立大学）；University of Florida, Gainesville, FL, USA（佛罗里达大学）；NEC Laboratories America, Princeton, NJ, USA（NEC美国实验室）；Clemson University, Clemson, SC, USA（克莱姆森大学）；Arizona State University, Tempe, AZ, USA（亚利桑那州立大学）
IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents
Massachusetts Institute of Technology（麻省理工学院）；University of Waterloo（滑铁卢大学）；Meta AI；AIRI（人工智能研究院）；École Polytechnique de Montréal（蒙特利尔理工学院）；Microsoft Research（微软研究院）
Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruction Following
Seoul National University（首尔大学）；Interdisciplinary Program in AI (IPAI), Seoul National University（首尔大学人工智能交叉项目）；AI Institute, Seoul National University（首尔大学人工智能研究院）
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts
School of Software Technology, Zhejiang University（浙江大学软件学院）；School of Computer Sciene and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机学院）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；The Hong Kong Polytechnic University（香港理工大学）；Zhiyuan Research Institute（智源研究院）；School of Computer Sciene and Technology, Xidian University（西安电子科技大学计算机学院）
Action- and Language-Conditioned Video Assessment for Embodied Control
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院电气工程学院）
$A^2$Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models
South China University of Technology（华南理工大学）；Shenzhen University（深圳大学）；Pazhou Laboratory（琶洲实验室）；Key Laboratory of Big Data and Intelligent Robot, Ministry of Education（教育部大数据与智能机器人重点实验室）；Shenzhen Graduate School, Peking University（北京大学深圳研究生院）；UMass Amherst（马萨诸塞大学阿默斯特分校）；Cisco Research（思科研究院）
