评估生成式搜索引擎在对抗性事实问题上的鲁棒性
Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions
AI总结：
本研究通过黑盒对抗性事实问题评估生成式搜索引擎的鲁棒性，发现其易受攻击且检索增强生成更易产生事实错误，强调部署前需严格评估。
摘要：
生成式搜索引擎有潜力改变人们在线获取信息的方式，但现有大型语言模型（LLMs）支持的生成式搜索引擎生成的回答可能并不总是准确的。然而，检索增强生成加剧了安全担忧，因为对手可能通过巧妙操纵声明中最脆弱的部分来成功规避整个系统。为此，我们提出在现实且高风险的环境中评估生成式搜索引擎的鲁棒性，其中对手仅具有黑盒系统访问权限，并试图欺骗模型返回不正确的回答。通过对各种生成式搜索引擎（如Bing Chat、PerplexityAI和YouChat）在不同查询上的全面人工评估，我们证明了对抗性事实问题在诱导错误回答方面的有效性。此外，与没有检索的LLMs相比，检索增强生成表现出更高的对事实错误的敏感性。这些发现凸显了这些系统的潜在安全风险，并强调了在部署前进行严格评估的必要性。
英文摘要：
Generative search engines have the potential to transform how people seek information online, but generated responses from existing large language models (LLMs)-backed generative search engines may not always be accurate. Nonetheless, retrieval-augmented generation exacerbates safety concerns, since adversaries may successfully evade the entire system by subtly manipulating the most vulnerable part of a claim. To this end, we propose evaluating the robustness of generative search engines in the realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning incorrect responses. Through a comprehensive human evaluation of various generative search engines, such as Bing Chat, PerplexityAI, and YouChat across diverse queries, we demonstrate the effectiveness of adversarial factual questions in inducing incorrect responses. Moreover, retrieval-augmented generation exhibits a higher susceptibility to factual errors compared to LLMs without retrieval. These findings highlight the potential security risks of these systems and emphasize the need for rigorous evaluation before deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
