Replicating human-level intelligence in the execution of embodied tasks remains challenging due to the unconstrained nature of real-world environments. Novel use of large language models (LLMs) for task planning seeks to address the previously intractable state/action space of complex planning tasks, but hallucinations limit their reliability, and thus, viability beyond a research context. Additionally, the prompt engineering required to achieve adequate system performance lacks transparency, and thus, repeatability. In contrast to LLM planning, symbolic planning methods offer strong reliability and repeatability guarantees, but struggle to scale to the complexity and ambiguity of real-world tasks. We introduce a new robotic planning method that augments LLM planners with symbolic planning oversight to improve reliability and repeatability, and provide a transparent approach to defining hard constraints with considerably stronger clarity than traditional prompt engineering. Importantly, these augmentations preserve the reasoning capabilities of LLMs and retain impressive generalization in open-world environments. We demonstrate our approach in simulated and real-world environments. On the ALFWorld planning benchmark, our approach outperforms current state-of-the-art methods, achieving a near-perfect 99% success rate. Deployment of our method to a real-world quadruped robot resulted in 100% task success compared to 50% and 30% for pure LLM and symbolic planners across embodied pick and place tasks. Our approach presents an effective strategy to enhance the reliability, repeatability and transparency of LLM-based robot planners while retaining their key strengths: flexibility and generalizability to complex real-world environments. We hope that this work will contribute to the broad goal of building resilient embodied intelligent systems.
LLM-Driven Self-Refinement for Embodied Drone Task Planning
School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
GraphThink: Graph-Enhanced LLM Thinking for Long-Horizon Embodied Task Planning
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Tsinghua University（清华大学）；The Hong Kong Polytechnic University（香港理工大学）
NEXUS: Learning Safety Constraints and Action Models for Safe and Robust Embodied Planning
School of Artificial Intelligence and Automation（人工智能与自动化学院）；Huazhong University of Science and Technology（华中科技大学）；School of Software（软件学院）；Henan University（河南大学）；Institute of Information Engineering（信息工程研究所）；Chinese Academy of Sciences（中国科学院）；School of Cyberspace Security（网络空间安全学院）；University of the Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）
Instruction-Augmented Long-Horizon Planning: Embedding Grounding Mechanisms in Embodied Mobile Manipulation
The Hong Kong Polytechnic University（香港理工大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）；Great Bay University（大湾区大学）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents
Department of Computer Science and Engineering（计算机科学与工程系）；Shanghai Jiao Tong University（上海交通大学）；Department of Computer Science（计算机科学系）；University of Georgia（佐治亚大学）；Shanghai AI Laboratory（上海人工智能实验室）
Using large language models for embodied planning introduces systematic safety risks
ETH Zurich（苏黎世联邦理工学院）；University College London（伦敦大学学院）；Stanford University（斯坦福大学）；Northwestern University（西北大学）；National University of Singapore（新加坡国立大学）
KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planning
Institute for Artificial Intelligence（人工智能研究院）；School of Integrated Circuits, Peking University（集成电路学院，北京大学）；Shenzhen Institute of Artificial Intelligence（深圳人工智能研究院）；Microsoft Research（微软研究院）；Beijing Advanced Innovation Center for Integrated Circuits（北京集成电路先进创新中心）
Recent works have shown how the reasoning capabilities of Large Language Models (LLMs) can be applied to domains beyond natural language processing, such as planning and interaction for robots. These embodied problems require an agent to understand many semantic aspects of the world: the repertoire of skills available, how these skills influence the world, and how changes to the world map back to the language. LLMs planning in embodied environments need to consider not just what skills to do, but also how and when to do them - answers that change over time in response to the agent's own choice…
Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Model
University of Chinese Academy of Sciences (UCAS)（中国科学院大学）；Tsinghua University（清华大学）；JD Explore Academy（京东探索学院）；ShanghaiTech University（上海科技大学）；Nanjing University（南京大学）
This work presents an embodied agent that can adapt its semantic segmentation network to new indoor environments in a fully autonomous way. Because semantic segmentation networks fail to generalize well to unseen environments, the agent collects images of the new environment which are then used for self-supervised domain adaptation. We formulate this as an informative path planning problem, and present a novel information gain that leverages uncertainty extracted from the semantic model to safely collect relevant data. As domain adaptation progresses, these uncertainties change over time and t…
LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Robotics
Automation and Control Institute, Faculty of Electrical Engineering, TU Wien（自动化与控制研究所，电气工程学院，维也纳技术大学）；AIT Austrian Institute of Technology GmbH, Center for Vision, Automation and Control（奥地利技术研究所（AIT）有限公司，视觉、自动化与控制中心）
Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruction Following
Seoul National University（首尔大学）；Interdisciplinary Program in AI (IPAI), Seoul National University（首尔大学人工智能交叉项目）；AI Institute, Seoul National University（首尔大学人工智能研究院）
