We present SENTINEL, a framework for formally evaluating the physical safety of foundation model (FM)-based embodied agents. SENTINEL is the first to provide multi-level safety evaluation across semantic interpretation, plan generation, and physical execution within a unified formal framework. Unlike prior methods that rely on heuristic rules or subjective FM judgments, SENTINEL grounds practical safety requirements in formal temporal logic (TL) semantics that can precisely specify state invariants, temporal dependencies, and timing constraints. It employs a multi-level verification pipeline where (i) at the semantic level, intuitive natural language safety requirements are formalized into TL formulas and the agent's understanding of these requirements is probed for alignment with the TL formulas; (ii) at the plan level, high-level action plans and subgoals generated by the agent are verified against the TL formulas to detect unsafe plans before execution; and (iii) at the trajectory level, multiple execution trajectories are merged into a computation tree and efficiently verified against physically-detailed TL specifications for a final safety check. We apply SENTINEL in VirtualHome and AI2-THOR, and formally evaluate multiple FM-based embodied agents against diverse safety requirements. Our experiments show that by grounding physical safety in temporal logic and applying verification methods across multiple levels, SENTINEL provides a rigorous foundation for systematically evaluating the safety of FM-based embodied agents in simulation-based physical environments, and can effectively expose potential safety violations in interpreting, planning, and executing the tasks.
Bridging VLMs and Embodied Intelligence with Deliberate Practice Policy Optimization
X-Humanoid;Imperial College London(帝国理工学院);Peking University(北京大学);University of Manchester(曼彻斯特大学);Westlake University(西湖大学);Fudan University(复旦大学);Waseda University(早稻田大学);Nvidia;Queen Mary University of London(伦敦大学玛丽女王学院);Celonis AI;Meta AI
Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation
UNIST InnoCORE AI-Space Solar Initiative(UNIST创新核心人工智能空间太阳能计划);Ulsan National Institute of Science and Technology (UNIST)(乌山国立科学技术研究院);Automation and Systems Research Institute(自动化与系统研究所);Department of Electrical and Computer Engineering(电气与计算机工程系);Interdisciplinary Program in Artificial Intelligence(人工智能跨学科项目);LG Electronics(LG电子)
ConEQsA: Concurrent and Asynchronous Embodied Questions Scheduling and Answering
Software Engineering Institute, East China Normal University(东华大学软件工程学院);Department of Computer Science, Yale University(耶鲁大学计算机科学系);School of Computer Science, The University of Sydney(悉尼大学计算机科学学院);College of Connected Computing, Vanderbilt University(范德比大学连接计算学院)
WorldSimProbe: Diagnosing Simulator Faithfulness in Action-Conditioned World Models for Embodied Manipulation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理国家重点实验室);EvoPhys AI(EvoPhys人工智能公司);Joy Future Academy, JD(京东探索研究院);The University of Sydney(悉尼大学);The Hong Kong University of Science and Technology(香港科技大学);Beijing Institute of Technology(北京理工大学)
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Georgia Institute of Technology(佐治亚理工学院);University of Texas at Dallas(德克萨斯大学达拉斯分校)
Managing Action Preconditions in Neuro-Symbolic RL: Three Placement Strategies for Embodied Agents
Institute of Distributed Intelligent Systems(分布式智能系统研究所);University of the Bundeswehr Munich(慕尼黑联邦国防军大学);School of Engineering(工程学院);The University of Western Australia(西澳大利亚大学)
OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows
The University of Hong Kong(香港大学);Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室);Nanyang Technological University(南洋理工大学);Nanjing University(南京大学);Shanghai Jiao Tong University(上海交通大学)
Collecting Interactive Multi-modal Datasets for Grounded Language Understanding
Microsoft Research(微软研究院);University of Waterloo(滑铁卢大学);Meta Research(Meta研究院);Moscow Institute of Physics Technology(莫斯科物理技术学院);Artificial Intelligence Research Institute(人工智能研究院)
SafeMind: Benchmarking and Mitigating Safety Risks in Embodied LLM Agents
Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences(脑启发认知AI实验室,自动化研究所,中国科学院);Beijing Key Laboratory of Safe AI and Superalignment(北京安全AI与超对齐关键实验室);Beijing Institute of AI Safety and Governance(北京AI安全与治理研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学);Long-term AI(长期AI)
Training Cross-Morphology Embodied AI Agents: From Practical Challenges to Theoretical Foundations
Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳人工智能与机器人研究院);Quantum Science Center of Guangdong-Hong Kong-Macau Greater Bay Area(粤港澳大湾区量子科学中心)
State Key Laboratory of Robotics and Intelligent Systems(机器人与智能系统国家重点实验室);Shenyang Institute of Automation(沈阳自动化研究所);Chinese Academy of Sciences(中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
LogicGuard: Improving Embodied LLM agents through Temporal Logic based Critics
Department of Mechanical Engineering, University of California at Santa Barbara(加州大学圣芭芭拉分校机械工程系);Department of Electrical and Computer Engineering, Michigan State University(密歇根州立大学电气与计算机工程系)
Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges
Shenzhen International Graduate School (SIGS), Tsinghua University(深圳国际研究生院(SIGS),清华大学);New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences(模式识别新实验室(NLPR),自动化研究所,中国科学院)
AEROS: A Single-Agent Operating Architecture with Embodied Capability Modules
School of Software, Harbin Institute of Technology(哈尔滨工业大学软件学院);School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院);School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus(赫瑞-沃森大学马来西亚分校数学与计算机科学学院);School of Future Science and Engineering, Soochow University(苏州大学未来科学与工程学院)
Federated Single-Agent Robotics: Multi-Robot Coordination Without Intra-Robot Multi-Agent Fragmentation
School of Software, Harbin Institute of Technology(哈尔滨工业大学软件学院);School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院);School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus(赫瑞-沃顿大学马来西亚校区数学与计算机科学学院);School of Future Science and Engineering, Soochow University(苏州大学未来科学与工程学院)