AI 中文总结
本研究针对视觉-语言-动作机器人面临的物理注意力劫持攻击，提出注意力引导语义破坏攻击方法，并开发结构感知鲁棒微调防御方法，大幅降低攻击失败率并提升真实机械臂操控成功率。
AI 中文摘要
视觉-语言-动作（VLA）策略可实现通用机器人操控，但其对物理世界攻击的鲁棒性仍较脆弱。本研究表明，可物理实现的对抗性补丁能通过触发一种被称为“策略关键动作-视觉注意力劫持”的机制，可靠引发故障；该机制中，动作条件注意力会从任务相关区域转移至局部补丁。为展示该威胁，我们提出注意力引导语义破坏（AGSD），这是一种经变换期望（EOT）优化的可打印补丁，可同时实现两点：一是将动作-视觉注意力集中于补丁，二是破坏视觉-语言语义对齐，从而产生强跨任务与跨架构迁移能力。为缓解此类攻击，我们引入结构感知鲁棒微调（SARF），这是一种无推理开销的防御方法，仅对视觉编码器进行微调，采用特征锚定、策略关键注意力修正，以及受限于语义相关区域的语言引导几何一致性。在LIBERO数据集上，SARF将OpenVLA在AGSD下的失败率从100%降至各测试套件的14.2%-56.8%（平均28.6%），同时保留干净样本性能；在真实PiPER机械臂上，其将AGSD下的平均成功率从23.0%提升至65.0%。这些结果凸显机制级鲁棒性是保障VLA机器人抵御物理注意力劫持的可行路径。
英文摘要
Vision-Language-Action (VLA) policies promise general robotic manipulation, but their robustness against physical-world attacks remains fragile. In particular, we show that physically realizable adversarial patches can reliably induce failures by triggering a mechanism we call policy-critical action-to-vision attention hijacking, where action-conditioned attention is diverted from task-relevant regions to a localized patch. To demonstrate the threat, we propose Attention-Guided Semantic Disruption (AGSD), an Expectation-over-Transformation (EOT) optimized printable patch that jointly (i) concentrates action-to-vision attention on the patch and (ii) disrupts vision-language semantic alignment, yielding strong cross-task and cross-architecture transfer. To mitigate such attacks, we introduce Structure-Aware Robust Fine-Tuning (SARF), a zero-inference-overhead defense that fine-tunes only the visual encoder using feature anchoring, policy-critical attention correction, and language-guided geometric consistency restricted to semantically relevant regions. On LIBERO, SARF reduces OpenVLA's failure rate under AGSD from 100% to 14.2%-56.8% (28.6% average) across suites while preserving clean performance, and on a real PiPER manipulator it improves average success under AGSD from 23.0% to 65.0%. These results highlight mechanism-level robustness as a practical path to securing VLA robots against physical attention hijacking.
CommentsAccepted to the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026)
