BEAT：通过对比触发学习对基于VLM的具身代理进行视觉后门攻击
BEAT: Visual Backdoor Attacks on VLM-based Embodied Agents via Contrastive Trigger Learning
AI总结：
BEAT通过对比触发学习在基于VLM的具身代理中实现视觉后门攻击，提升后门激活精度至39%。
摘要：
最近视觉语言模型（VLMs）的进步推动了具身代理的发展，使代理能够直接从视觉输入中进行感知、推理和规划任务导向的动作。然而，这种视觉驱动的具身代理开辟了新的攻击面：视觉后门攻击，其中代理在场景中出现视觉触发时表现正常，然后持续执行攻击者指定的多步骤策略。我们介绍了BEAT，这是首个框架，通过环境中的物体作为触发器，将此类视觉后门注入到基于VLM的具身代理中。与文本触发器不同，物体触发器在不同视角和光照下表现出广泛变化，使可靠植入变得困难。BEAT通过（1）构建跨越多样场景、任务和触发器放置的训练集，使代理接触触发器变化，以及（2）引入两阶段训练方案，首先应用监督微调（SFT），然后我们新颖的对比触发学习（CTL）。CTL将触发器辨别作为触发器存在与触发器不存在输入之间的偏好学习，明确地锐化决策边界，以确保精确的后门激活。在各种具身代理基准和VLMs上，BEAT实现了高达80%的攻击成功率，同时保持强大的良性任务性能，并能可靠地推广到分布外的触发器放置。值得注意的是，与朴素SFT相比，CTL在有限的后门数据下将后门激活准确性提高了39%。这些发现揭示了VLM基于具身代理中的关键但未探索的安全风险，突显了在现实世界部署之前需要强大的防御措施的必要性。
英文摘要：
Recent advances in Vision-Language Models (VLMs) have propelled embodied agents by enabling direct perception, reasoning, and planning task-oriented actions from visual inputs. However, such vision-driven embodied agents open a new attack surface: visual backdoor attacks, where the agent behaves normally until a visual trigger appears in the scene, then persistently executes an attacker-specified multi-step policy. We introduce BEAT, the first framework to inject such visual backdoors into VLM-based embodied agents using objects in the environments as triggers. Unlike textual triggers, object triggers exhibit wide variation across viewpoints and lighting, making them difficult to implant reliably. BEAT addresses this challenge by (1) constructing a training set that spans diverse scenes, tasks, and trigger placements to expose agents to trigger variability, and (2) introducing a two-stage training scheme that first applies supervised fine-tuning (SFT) and then our novel Contrastive Trigger Learning (CTL). CTL formulates trigger discrimination as preference learning between trigger-present and trigger-free inputs, explicitly sharpening the decision boundaries to ensure precise backdoor activation. Across various embodied agent benchmarks and VLMs, BEAT achieves attack success rates up to 80%, while maintaining strong benign task performance, and generalizes reliably to out-of-distribution trigger placements. Notably, compared to naive SFT, CTL boosts backdoor activation accuracy up to 39% under limited backdoor data. These findings expose a critical yet unexplored security risk in VLM-based embodied agents, underscoring the need for robust defenses before real-world deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
