揭示视觉语言模型的可解释性故障模式
Revealing Interpretable Failure Modes of VLMs
AI总结：
本文提出REVELIO框架，通过系统性探索揭示VLMs在自动驾驶和室内机器人领域中的可解释性故障模式，发现模型在空间定位和安全威胁识别上的缺陷。
摘要：
视觉语言模型（VLMs）因其广泛推理能力和最小任务特定工程的泛化能力而越来越多地应用于安全关键应用。尽管这些优势，它们在特定现实情况中可能表现出灾难性故障，构成故障模式。我们引入REVELIO，一个用于系统性揭示VLMs可解释性故障模式的框架。我们定义故障模式为由可解释、领域相关的概念（如行人接近度或恶劣天气条件）组成的组合，使得目标VLM始终表现错误。识别此类故障需要搜索一个指数级大的离散组合空间。为解决这一挑战，REVELIO结合了两种搜索过程：一种是多样性意识的束搜索，能够高效地映射故障景观，另一种是高斯过程汤普森采样策略，能够更广泛地探索复杂的故障模式。我们应用REVELIO到自动驾驶和室内机器人领域，揭示了当前最先进的VLMs中之前未报告的漏洞。在驾驶环境中，模型经常表现出空间定位弱和未能考虑主要障碍物，导致模拟碰撞的推荐。在室内机器人任务中，VLMs要么错过安全威胁，要么行为过于保守，产生假警报并降低操作效率。通过识别结构化和可解释的故障模式，REVELIO提供了可操作的见解，以支持针对性的VLM安全改进。
英文摘要：
Vision-Language Models (VLMs) are increasingly used in safety-critical applications because of their broad reasoning capabilities and ability to generalize with minimal task-specific engineering. Despite these advantages, they can exhibit catastrophic failures in specific real-world situations, constituting failure modes. We introduce REVELIO, a framework for systematically uncovering interpretable failure modes in VLMs. We define a failure mode as a composition of interpretable, domain-relevant concepts-such as pedestrian proximity or adverse weather conditions-under which a target VLM consistently behaves incorrectly. Identifying such failures requires searching over an exponentially large discrete combinatorial space. To address this challenge, REVELIO combines two search procedures: a diversity-aware beam search that efficiently maps the failure landscape, and a Gaussian-process Thompson Sampling strategy that enables broader exploration of complex failure modes. We apply REVELIO to autonomous driving and indoor robotics domains, uncovering previously unreported vulnerabilities in state-of-the-art VLMs. In driving environments, the models often demonstrate weak spatial grounding and fail to account for major obstructions, leading to recommendations that would result in simulated crashes. In indoor robotics tasks, VLMs either miss safety hazards or behave excessively conservatively, producing false alarms and reducing operational efficiency. By identifying structured and interpretable failure modes, REVELIO offers actionable insights that can support targeted VLM safety improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
