FabriMAE：我相信自己吗？基于马尔可夫注意力熵的视觉-语言-动作模型动作自评估

中文摘要 AI 辅助

视觉-语言-动作模型（VLAs）将视觉感知、语言指令和动作生成为端到端策略，适用于异构架构。然而，让VLAs在无外部监督的情况下自评估其动作生成的可靠性仍是重大挑战。现有方法要么依赖专家标注，要么仅从输出统计量估计不确定性，很大程度上忽略了内部信号。本研究观察到，在异构VLAs中，内部视觉模态熵在成功与失败任务间存在一致差异。尽管VLAs的动作生成架构不同，但它们共享由视觉感知、语言指令和状态输入演化而来的共同潜在动作生成抽象，我们将其形式化为条件生成马尔可夫链。基于此形式化，我们提出MAE（马尔可夫注意力熵），一种直接将内部注意力信号转换为架构感知可靠性分数的自评估框架，并引入LIBERO-Reflect，一个包含四个子集共4000个回合的基准，其中2000个为标准回合，2000个为具有挑战性的回合。在异构VLA架构和多样场景下的大量实验表明，MAE在AUPR、AUROC和FPR@95指标上始终优于现有最优基线。我们进一步将FabriMAE实例化为无验证器的测试时动作选择，显示MAE引导的多重采样在LIBERO-Plus上提升了PI系列的鲁棒性，且观察到的运行时开销较小。

英文摘要

Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies across heterogeneous architectures. However, enabling VLAs to self-evaluate their action generation reliability without external supervision remains a major challenge. Existing methods either rely on expert annotations or estimate uncertainty only from output statistics, largely ignoring internal signals. In this work, we observe that internal visual modality entropy exhibits consistent distinctions between successful and failed tasks across heterogeneous VLAs. Although VLAs' architectures differ in their action generation, we show that they share a common latent action generation abstraction evolving under visual perception, language instruction, and State Input, which we formulate as a Conditional Generative Markov Chain. Based on this formulation, we propose MAE (Markov Attention Entropy), a self-evaluation framework that directly converts internal attention signals into architecture-aware reliability scores, and introduce LIBERO-Reflect, a 4,000-episode benchmark combining 2,000 standard episodes and 2,000 challenging episodes across four subsets. Extensive experiments across heterogeneous VLA architectures and diverse scenarios show that MAE consistently outperforms state-of-the-art baselines on AUPR, AUROC, and FPR@95.

发表机构

• National University of Singapore（新加坡国立大学）
• Ludwig Maximilian University of Munich（慕尼黑大学）
• Munich Center for Machine Learning（慕尼黑机器学习中心）
• Amazon（亚马逊）
• East China University of Science and Technology（华东理工大学）
• Mese Technology Limited Co., Ltd.（迈泽科技有限公司）
• FabriX team at Youibot Robotics Co., Ltd.（优必科技有限公司FabriX团队）

机构由 AI 辅助整理，请以论文原文为准。