机器人崩溃:针对基于VLM的机器人操控的供应链后门攻击
Robot Collapse: Supply Chain Backdoor Attacks Against VLM-based Robotic Manipulation
AI总结:
本文提出TrojanRobot框架,通过在模块化机器人策略中嵌入恶意模块,操控LLM到VLM路径,验证了在18个真实任务和4个VLM上的优越性。
摘要:
机器人操控策略越来越多地依赖大型语言模型(LLMs)和视觉-语言模型(VLMs),利用其理解和感知能力。最近,针对机器人操控的推理时间攻击已被广泛研究,但针对机器人策略模型供应链安全的后门攻击仍鲜有探索。为填补这一空白,我们提出了TrojanRobot,一种用于模型供应链攻击场景的后门注入框架,通过后门关系在模块化机器人策略中嵌入恶意模块,以操纵LLM到VLM路径并破坏系统。我们的基本设计将此模块作为经过后门微调的VLM。为进一步提升攻击性能,我们提出了一种prime方案,引入了LVLM-as-a-backdoor的概念,利用上下文指令学习(ICIL)通过后门系统提示引导大型视觉-语言模型(LVLM)的行为。此外,我们开发了三种类型的prime攻击,即排列、停滞和故意攻击,实现了灵活的后门攻击效果。在18个真实世界操控任务和4个VLM上的广泛物理世界和模拟器实验验证了所提TrojanRobot的优越性。
英文摘要:
Robotic manipulation policies are increasingly empowered by \textit{large language models} (LLMs) and \textit{vision-language models} (VLMs), leveraging their understanding and perception capabilities. Recently, inference-time attacks against robotic manipulation have been extensively studied, yet backdoor attacks targeting model supply chain security in robotic policies remain largely unexplored. To fill this gap, we propose \texttt{TrojanRobot}, a backdoor injection framework for model supply chain attack scenarios, which embeds a malicious module into modular robotic policies via backdoor relationships to manipulate the LLM-to-VLM pathway and compromise the system. Our vanilla design instantiates this module as a backdoor-finetuned VLM. To further enhance attack performance, we propose a prime scheme by introducing the concept of \textit{LVLM-as-a-backdoor}, which leverages \textit{in-context instruction learning} (ICIL) to steer \textit{large vision-language model} (LVLM) behavior through backdoored system prompts. Moreover, we develop three types of prime attacks, \textit{permutation}, \textit{stagnation}, and \textit{intentional}, achieving flexible backdoor attack effects. Extensive physical-world and simulator experiments on 18 real-world manipulation tasks and 4 VLMs verify the superiority of proposed \texttt{TrojanRobot}
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。