RESCUE：该警报是否值得采取行动？VLA运行时监控器的反事实评估
RESCUE: Is This Alarm Worth Acting On? Counterfactual Evaluation of VLA Runtime Monitors
AI总结：
本文提出RESCUE反事实框架，评估VLA运行时监控器警报的干预价值，经LIBERO数据集实验发现监控器价值取决于可用干预与部署工作负载。
摘要：
机器人策略的运行时故障监控器通常通过区分失败与成功执行的准确性来评估，但准确的警报可能因到达过晚而无法恢复，不必要的干预也可能中断原本会成功的执行。本文提出RESCUE，这是一种用于评估监控器警报是否能实现有用干预的反事实框架。通过在执行检查点恢复模拟器和策略状态，RESCUE可比较替代行动、识别满足指定恢复标准的最新检查点，并评估基于警报采取行动是否能降低预期任务完成成本。我们在LIBERO-Spatial和LIBERO-Plus数据集上，使用OpenVLA-OFT和π₀.5模型，通过128340个反事实分支对RESCUE进行评估。可恢复性取决于状态-干预对，而非仅机器人状态：在τ=0.5时，66个OpenVLA-OFT/LIBERO-Plus失败案例中，14个在仅继续执行的情况下存在可恢复检查点，而在继续执行或脚本恢复的情况下有34个；在其他条件下，恢复会降低可恢复性。在主要的强策略条件下，良好的检测分数并不能证明干预成本更低。在评估的停止-重置模型下，即使是理想化的检测器，在不必要的重置超过提前停止失败的节省之前，也只能承受2.0%-3.3%的误报率。这些结果表明，监控器的价值取决于可用干预措施及其部署的工作负载。
英文摘要：
Runtime failure monitors for robot policies are typically evaluated by how accurately they distinguish failed from successful executions. Yet an accurate alarm may arrive too late for recovery, while an unnecessary intervention may disrupt an execution that would otherwise succeed. We introduce RESCUE, a counterfactual framework for evaluating whether a monitor's alarms enable useful interventions. By restoring simulator and policy states at execution checkpoints, RESCUE compares alternative actions, identifies the latest checkpoint satisfying a specified recovery criterion, and evaluates whether acting on alarms reduces expected task-completion cost. We evaluate RESCUE with OpenVLA-OFT and $π_{0.5}$ over LIBERO-Spatial and LIBERO-Plus using 128,340 counterfactual branches. Recoverability depends on the state--intervention pair rather than on robot state alone: at $τ=0.5$, 14 of 66 OpenVLA-OFT/LIBERO-Plus failures contain a recoverable checkpoint under continuation, compared with 34 of 66 under continuation or scripted recovery, while recovery reduces recoverability in other conditions. On the primary strong-policy conditions, favourable detection scores do not establish lower intervention cost. Under the evaluated stop-and-reset model, even an idealised detector can afford only a 2.0--3.3% false-alarm rate before unnecessary resets outweigh the savings from stopping failures early. These results show that monitor value depends jointly on the intervention available and the workload in which it is deployed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
