arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.09546cs.ROcs.AI
收藏

大型语言模型(LLM)在城市环境导航中的安全性如何?

How Secure Are Large Language Models (LLMs) for Navigation in Urban Environments?

Embodied AI and Robotics (AIR) Lab, New York University, New York, USA and NYUAD Center for Artificial Intelligence and Robotics, New York University Abu Dhabi(纽约大学Embodied AI和Robotics实验室及纽约大学阿布扎比分校人工智能与机器人中心);School of Software, Tsinghua University, Beijing, P. R. China(清华大学软件学院);Harvard AI and Robotics Lab, Harvard University, Boston, USA(哈佛大学人工智能与机器人实验室);NYUAD Center for Artificial Intelligence and Robotics, New York University Abu Dhabi(纽约大学阿布扎比分校人工智能与机器人中心)

Congcong Wen, Jiazhao Liang, Shuaihang Yuan, Hao Huang, Geeta Chandra Raju Bethala, Yu-Shen Liu, Mengyu Wang, Anthony Tzes, Yi Fang

AI总结:

本文首次探索基于LLM的城市导航模型的安全漏洞,提出导航提示插入与替换两类攻击,实验显示性能显著下降,并初步提出NPE防御策略以增强系统安全性。

摘要:

在机器人与自动化领域,基于大型语言模型(LLM)的导航系统近期展示了令人瞩目的性能。然而,这些系统的安全性问题受到的关注相对较少。本文率先探索了基于LLM的导航模型在城市场景中的脆弱性,鉴于该技术在自动驾驶、物流和应急服务中的广泛应用,这一领域至关重要。具体而言,我们提出了一种新颖的导航提示攻击(Navigational Prompt Attack),通过扰动原始导航提示来操纵基于LLM的导航模型,导致其产生错误行为。根据扰动方式的不同,我们的攻击分为两类:导航提示插入(NPI)攻击和导航提示替换(NPS)攻击。我们在一个采用多种LLM进行推理的基于LLM的导航模型上进行了全面实验。基于Touchdown和Map2Seq街景数据集,在少样本学习和微调两种配置下,我们的结果表明,在面对白盒和黑盒攻击时,七个指标均出现显著性能下降。此外,我们的攻击可以轻松扩展到其他基于LLM的导航模型,并取得同样有效的效果。这些发现凸显了所提出攻击的泛化性和可迁移性,强调了增强基于LLM的导航系统安全性的必要性。作为初步对策,我们提出了导航提示工程(NPE)防御策略,该策略聚焦于导航相关关键词,以降低对抗攻击的影响。虽然初步结果表明该策略提升了导航安全性,但更广泛的研究社区仍需开发更强的防御方法,以有效应对这些系统在现实世界中面临的挑战。

英文摘要:

In the field of robotics and automation, navigation systems based on Large Language Models (LLMs) have recently demonstrated impressive performance. However, the security aspects of these systems have received relatively less attention. This paper pioneers the exploration of vulnerabilities in LLM-based navigation models in urban outdoor environments, a critical area given the widespread application of this technology in autonomous driving, logistics, and emergency services. Specifically, we introduce a novel Navigational Prompt Attack that manipulates LLM-based navigation models by perturbing the original navigational prompt, leading to incorrect actions. Based on the method of perturbation, our attacks are divided into two types: Navigational Prompt Insert (NPI) Attack and Navigational Prompt Swap (NPS) Attack. We conducted comprehensive experiments on an LLM-based navigation model that employs various LLMs for reasoning. Our results, derived from the Touchdown and Map2Seq street-view datasets under both few-shot learning and fine-tuning configurations, demonstrate notable performance declines across seven metrics in the face of both white-box and black-box attacks. Moreover, our attacks can be easily extended to other LLM-based navigation models with similarly effective results. These findings highlight the generalizability and transferability of the proposed attack, emphasizing the need for enhanced security in LLM-based navigation systems. As an initial countermeasure, we propose the Navigational Prompt Engineering (NPE) Defense strategy, which concentrates on navigation-relevant keywords to reduce the impact of adversarial attacks. While initial findings indicate that this strategy enhances navigational safety, there remains a critical need for the wider research community to develop stronger defense methods to effectively tackle the real-world challenges faced by these systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑