ProTracer：机器人操作中本体感觉引导的故障诊断
ProTracer: Proprioception-Guided Failure Diagnosis in Robot Manipulation
浏览论文内容
中文总结 AI 辅助
本文提出ProTracer，一种无需训练的框架，结合视觉语言模型与本体感觉信号，实现机器人操作故障检测、分类、解释及起始定位，并引入FailTime基准验证其有效性。
中文摘要 AI 辅助
本文提出了一个用于机器人操作故障分析的全面框架，该框架包括二元故障检测、故障分类、解释生成以及额外的故障起始定位能力，其目标是识别机器人执行偏离有效任务完成轨迹的最早时刻，并最终导致任务失败。为了解决这些任务，我们提出了ProTracer，一个无需训练的框架，该框架利用现有的视觉语言模型（VLMs）结合本体感觉信号进行故障分析。我们的方法使用本体感觉动力学来识别时间上有信息的动作边界，并将更丰富的机器人状态信号转换为结构化的自然语言描述，这些描述可以与视觉观察一起由VLM进行联合分析。这种设计将本体感觉信号的时间精度与现代VLM的多模态推理能力相结合，而无需额外的模型训练。我们进一步引入了FailTime，一个具有同步视觉和本体感觉观测的基准，用于评估传统故障诊断任务以及故障起始定位。实验表明，ProTracer在传统故障诊断任务和新引入的故障起始定位任务上均取得了强劲的性能，突显了本体感觉推理对于细粒度时间故障分析的重要性。
英文摘要
This paper presents a comprehensive framework for robot manipulation failure analysis that includes binary failure detection, failure categorization, explanation generation, and the additional capability of failure onset localization, which aims to identify the earliest moment at which a robot execution deviates from a valid task-completion trajectory and is ultimately followed by task failure. To address these tasks, we propose ProTracer, a training-free framework that leverages existing Vision-Language Models (VLMs) together with proprioceptive signals for failure analysis. Our method uses proprioceptive dynamics to identify temporally informative action boundaries and converts richer robot-state signals into structured natural-language descriptions that can be jointly analyzed together with visual observations by the VLM. This design combines the temporal precision of proprioceptive signals with the multimodal reasoning capabilities of modern VLMs without requiring additional model training. We further introduce FailTime, a benchmark with synchronized visual and proprioceptive observations for evaluating conventional failure diagnosis tasks as well as failure onset localization. Experiments demonstrate that ProTracer achieves strong performance across both conventional failure diagnosis tasks and the newly introduced failure onset localization task, highlighting the importance of proprioceptive reasoning for fine-grained temporal failure analysis.
发表机构
- Australian Institute for Machine Learning（澳大利亚机器学习研究所）
- School of Allied Health and Human Performance, Adelaide University（阿德莱德大学联合健康与人类绩效学院）
机构由 AI 辅助整理，请以论文原文为准。
