arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.29699cs.CVcs.AIcs.RO
收藏

视觉分布偏移下OpenVLA故障的早期预警信号

Early Warning Signals for OpenVLA Failure under Visual Distribution Shift

New York University(纽约大学)

Dipesh Tharu Mahato, Rachel Ren

AI总结:

研究在视觉分布偏移下,OpenVLA内部激活是否包含可线性解码的近期任务失败信息,通过LIBERO操作实验发现第16层逻辑探针在遮挡条件下预测失败AUROC达0.972。

摘要:

视觉语言动作模型将感知、语言基础和控制在单一策略中结合,但当视觉条件发生变化时,其故障难以诊断。我们测试了OpenVLA前馈激活是否包含关于LIBERO操作滚动中近期任务失败的线性可解码信息。策略在整个过程中固定。我们在执行过程中记录内部激活,并在收集滚动后拟合轻量级监视器。遮挡是主要的受控压力测试。它将OpenVLA的成功率从每个条件100个回合的57%降低到17%。在这种偏移下,第16层的逻辑探针在预测15步范围内的失败时达到AUROC 0.972和AUPRC 0.352。它优于均值差异方向和动作不一致基线。稀疏层扫描发现不同深度的可解码性不均匀:第16层在测试层中最强,第8层仍然有信息,第10层较弱。为了检查监视器是否仅仅是遮挡检测器,我们还评估了颜色偏移和相机抖动而不重新拟合。颜色偏移在这种情况下不会产生失败,因此它是一个良性控制而非失败基准。相机抖动确实导致失败,并且遮挡训练的监视器仍然优于随机。结果是有意限制的:OpenVLA内部状态在受控感知偏移下包含与失败相关的结构,但这些实验并未建立因果机制、任务外推泛化或可部署的恢复系统。

英文摘要:

Visual shifts can cause a vision-language-action policy to fail after initially plausible behavior. We ask whether OpenVLA's internal activations contain signals associated with the steps before failure. We freeze the policy, record one MLP activation per LIBERO-10 step, and fit two linear monitors. Occlusion reduces task success from $57\%$ to $17\%$. Within failed matched-reset trajectories, a layer-16 logistic probe attains AUROC $0.972$ and AUPRC $0.352$, whereas action disagreement attains AUROC $0.496$. Without refitting, the occlusion-trained probe reaches AUROC $0.689$ on failed camera-jitter episodes. In a calibration check, however, the same layer-16 monitor averages 3.32 warning onsets per clean episode. This contrast shows that strong retrospective discrimination does not imply operationally quiet warning behavior. Because fitting and evaluation share tasks, resets, and seed, these results establish retrospective separability rather than prediction on independent episodes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑