如果你在等待一个信号……那可能不是它！减轻视觉注入对视觉语言代理系统信任边界混淆的影响
If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems
AI总结：
研究探讨了视觉注入对视觉语言代理系统信任边界的影响，通过双意图数据集和评估框架发现现有模型在平衡合法信号与误导信号方面存在不足，提出多代理防御框架以提升系统鲁棒性。
摘要：
最近基于大视觉语言模型（LVLM）的具身视觉语言代理系统（VLAS）的进步，使AI系统能够感知和推理真实世界场景。在此背景下，环境信号如交通灯是必要的内带信号，能够并应影响代理行为。然而，类似的信号也可能被人为制造成误导性的视觉注入，覆盖用户意图并带来安全风险。这种双重性创造了一个根本性的挑战：代理必须对合法的环境信号做出反应，同时保持对误导信号的鲁棒性。我们称这种紧张为信任边界混淆。为了研究这种行为，我们设计了一个双意图数据集和评估框架，通过该框架展示了当前基于LVLM的代理在平衡这一权衡方面不可靠，要么忽略有用信号，要么跟随有害信号。我们系统地评估了7个LVLM代理在多个具身设置下，针对基于结构和噪声的视觉注入进行评估。为了解决这些漏洞，我们提出了一种多代理防御框架，将感知与决策分离，以动态评估视觉输入的可靠性。我们的方法显著减少了误导行为，同时保持正确响应，并在对抗扰动下提供鲁棒性保证。评估框架和成果的代码已发布在https://anonymous.4open.science/r/Visual-Prompt-Inject。
英文摘要：
Recent advances in embodied Vision-Language Agentic Systems (VLAS), powered by large vision-language models (LVLMs), enable AI systems to perceive and reason over real-world scenes. Within this context, environmental signals such as traffic lights are essential in-band signals that can and should influence agent behavior. However, similar signals could also be crafted to operate as misleading visual injections, overriding user intent and posing security risks. This duality creates a fundamental challenge: agents must respond to legitimate environmental cues while remaining robust to misleading ones. We refer to this tension as trust boundary confusion. To study this behavior, we design a dual-intent dataset and evaluation framework, through which we show that current LVLM-based agents fail to reliably balance this trade-off, either ignoring useful signals or following harmful ones. We systematically evaluate 7 LVLM agents across multiple embodied settings under both structure-based and noise-based visual injections. To address these vulnerabilities, we propose a multi-agent defense framework that separates perception from decision-making to dynamically assess the reliability of visual inputs. Our approach significantly reduces misleading behaviors while preserving correct responses and provides robustness guarantees under adversarial perturbations. The code of the evaluation framework and artifacts are made available at https://anonymous.4open.science/r/Visual-Prompt-Inject.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
