arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.12405cs.ROcs.LG
收藏

自适应多任务视觉语言模型用于机器人故障检测与推理

Self-Refining Vision Language Model for Robotic Failure Detection and Reasoning

UT Austin(德克萨斯大学);Amazon Robotics(亚马逊机器人技术);Carnegie Mellon University(卡内基梅隆大学)

Carl Qi, Xiaojie Wang, Silong Yong, Stephen Sheng, Huitan Mao, Sriram Srinivasan, Manikantan Nambi, Amy Zhang, Yesh Dattatreya

AI总结:

ARMOR是一种自适应多任务视觉语言模型,通过多任务自 refinement 过程提升机器人故障检测与推理性能,实现故障检测率提升30%和推理表现提升100%。

摘要:

对故障进行推理对于构建可靠且可信赖的机器人系统至关重要。先前的方法要么将故障推理视为一个封闭集分类问题,要么假设能够获得大量的人类注释。现实世界中的故障通常是微妙的、组合的且难以枚举,而丰富的推理标签获取成本很高。我们通过引入ARMOR:自适应轮次多任务模型来解决这个问题。我们将检测和推理作为多任务自 refinement 过程进行建模,其中模型会根据过去的输出迭代地预测检测结果和自然语言推理。在训练过程中,ARMOR 从异构监督中学习——大规模稀疏二元标签和小规模丰富的推理注释——通过结合离线和在线模仿学习进行优化。在推理过程中,ARMOR 生成多个细化轨迹,并通过自信心度量选择最自信的预测。在多样化的环境中进行实验表明,ARMOR 通过在故障检测率上比先前方法提高高达30%,并在通过LLM模糊匹配分数上达到100%的推理表现,证明了其在异构监督和超出预定义故障模式的开放性推理中的鲁棒性。我们还在网站上提供了额外的可视化:https://sites.google.com/utexas.edu/armor

英文摘要:

Reasoning about failures is crucial for building reliable and trustworthy robotic systems. Prior approaches either treat failure reasoning as a closed-set classification problem or assume access to ample human annotations. Failures in the real world are typically subtle, combinatorial, and difficult to enumerate, whereas rich reasoning labels are expensive to acquire. We address this problem by introducing ARMOR: Adaptive Round-based Multi-task mOdel for Robotic failure detection and reasoning. We formulate detection and reasoning as a multi-task self-refinement process, where the model iteratively predicts detection outcomes and natural language reasoning conditioned on past outputs. During training, ARMOR learns from heterogeneous supervision - large-scale sparse binary labels and small-scale rich reasoning annotations - optimized via a combination of offline and online imitation learning. At inference time, ARMOR generates multiple refinement trajectories and selects the most confident prediction via a self-certainty metric. Experiments across diverse environments show that ARMOR achieves state-of-the-art performance by improving over the previous approaches by up to 30% on failure detection rate and up to 100% in reasoning measured through LLM fuzzy match score, demonstrating robustness to heterogeneous supervision and open-ended reasoning beyond predefined failure modes. We provide additional visualizations on our website: https://sites.google.com/utexas.edu/armor

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑