Reasoning about failures is crucial for building reliable and trustworthy robotic systems. Prior approaches either treat failure reasoning as a closed-set classification problem or assume access to ample human annotations. Failures in the real world are typically subtle, combinatorial, and difficult to enumerate, whereas rich reasoning labels are expensive to acquire. We address this problem by introducing ARMOR: Adaptive Round-based Multi-task mOdel for Robotic failure detection and reasoning. We formulate detection and reasoning as a multi-task self-refinement process, where the model iteratively predicts detection outcomes and natural language reasoning conditioned on past outputs. During training, ARMOR learns from heterogeneous supervision - large-scale sparse binary labels and small-scale rich reasoning annotations - optimized via a combination of offline and online imitation learning. At inference time, ARMOR generates multiple refinement trajectories and selects the most confident prediction via a self-certainty metric. Experiments across diverse environments show that ARMOR achieves state-of-the-art performance by improving over the previous approaches by up to 30% on failure detection rate and up to 100% in reasoning measured through LLM fuzzy match score, demonstrating robustness to heterogeneous supervision and open-ended reasoning beyond predefined failure modes. We provide additional visualizations on our website: https://sites.google.com/utexas.edu/armor
How to Utilize Failure Demo Data?: Effective Data Selection for Imitation Learning Using Distribution Differences in Attention Mechanism
Faculty of Science and Engineering, Waseda University, Tokyo, Japan(科学与工程学部,早稻田大学,东京,日本);Physical AI Laboratory, Fujitsu Limited, Kanagawa, Japan(物理人工智能实验室, Fujitsu 有限,神奈川,日本)
World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulation
The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Zhongguancun Academy(中关村学院);Beijing Zhongke Huiling Robot Technology Co(北京中科创联机器人技术有限公司);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系)
Results of the Survey: Failures in Robotics and Intelligent Systems
Research Institute for Cognition and Robotics (CoR-Lab)(认知与机器人研究所);Center of Excellence Cognitive Interaction Technology (CITEC)(认知交互技术卓越中心);Bielefeld University(比勒费尔德大学)
AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation
NVIDIA(英伟达);University of Washington(华盛顿大学);Universidad Católica San Pablo(圣巴勃罗天主教大学);MIT(麻省理工学院);Nanyang Technological University(南洋理工大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所)
DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation
University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Carnegie Mellon University(卡内基梅隆大学);Shanghai Jiao Tong University(上海交通大学);Amazon AWS AI(亚马逊AWS人工智能)
FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models
Nanyang Technological University(南洋理工大学);Centre for Frontier AI Research, A*STAR(A*STAR前沿人工智能研究中心);Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学)
PhysPose: Refining 6D Object Poses with Physical Constraints
Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague(布拉格捷克理工大学捷克信息学、机器人学与控制论学院);Faculty of Electrical Engineering, Czech Technical University in Prague(布拉格捷克理工大学电气工程学院);Inria - Département d’Informatique de l’École normale supérieure, PSL Research University(法国国家信息与自动化研究所 - 巴黎高等师范学院信息学系,PSL大学)
LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models
School of Information and Communication Engineering(信息与通信工程学院);Dalian University of Technology(大连理工大学);Beta Infinity(贝塔无限);Nanyang Technological University(南洋理工大学);Beijing Jiaotong University(北京交通大学)
Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection
School of Software, Beihang University(北京航空航天大学软件学院);School of Computer Science, Peking University(北京大学计算机学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Galbot(盖尔博特(Galbot))
ReTrace: Cross-Stage Failure Attribution for Self-Improving Robotic Manipulation
College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机与人工智能学院);Institute of Trustworthy Embodied AI, Fudan University(复旦大学可信具身人工智能研究院);Singapore Management University(新加坡管理大学)
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts
School of Software Technology, Zhejiang University(浙江大学软件学院);School of Computer Sciene and Technology, Hangzhou Dianzi University(杭州电子科技大学计算机学院);China Mobile (Zhejiang) Research & Innovation Institute(中国移动(浙江)研究院);State Key Lab of CAD&CG, Zhejiang University(浙江大学CAD&CG国家重点实验室);The Hong Kong Polytechnic University(香港理工大学);Zhiyuan Research Institute(智源研究院);School of Computer Sciene and Technology, Xidian University(西安电子科技大学计算机学院)
FLARE: A Failure-Aware Framework for Autonomous Correction and Recovery in Visual-Language Robotic Manipulation
The Chinese University of Hong Kong(香港中文大学);Centre for Perceptual and Interactive Intelligence(感知与交互智能中心);Duke University(杜克大学);Sun Yat-sen University(中山大学);TengenX(天工科技);Tencent Robotics X(腾讯Robotics X实验室);Shenzhen Loop Area Institute(深圳河套学院);Guangdong Key Laboratory of Big Data Analysis and Processing(广东省大数据分析与处理重点实验室)
Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics
Zhejiang University(浙江大学);Beijing Zhongguancun Academy(北京中关村学院);University of Science and Technology of China(中国科学技术大学);Eastern Institute of Technology(东方理工学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
No Plan, Yet Human: A Reactive Robotics Model Predicts Human Planning Failures on a Clinical Task
Robotics and Biology Laboratory, Technische Universität Berlin, Germany(技术大学柏林机器人与生物学实验室,德国);Science of Intelligence, Research Cluster of Excellence, Berlin, Germany(智能科学,卓越研究集群,柏林,德国);Robotics Institute Germany(德国机器人研究所)