SafeArena：评估自主网络智能体的安全性
SafeArena: Evaluating the Safety of Autonomous Web Agents
AI总结：
SafeArena是首个评估网络智能体恶意滥用的基准，包含500个安全与有害任务，并通过风险评估框架揭示GPT-4o等主流模型对有害请求的高遵从率，凸显安全对齐的紧迫性。
摘要：
基于大语言模型（LLM）的智能体在解决基于网络的任务方面正变得越来越熟练。随着这种能力的提升，其被滥用于恶意目的的风险也随之增加，例如在网络论坛上发布错误信息或在网站上销售非法物品。为了评估这些风险，我们提出了SafeArena，这是第一个专注于网络智能体故意滥用行为的基准测试。SafeArena包含分布在四个网站上的250个安全任务和250个有害任务。我们将有害任务分为五个伤害类别——错误信息、非法活动、骚扰、网络犯罪和社会偏见，旨在评估网络智能体的现实滥用场景。我们在该基准上评估了领先的基于LLM的网络智能体，包括GPT-4o、Claude-3.5 Sonnet、Qwen-2-VL 72B和Llama-3.2 90B。为了系统性地评估它们对有害任务的易感性，我们引入了智能体风险评估框架，该框架将智能体行为划分为四个风险等级。我们发现智能体对恶意请求的遵从程度令人惊讶，其中GPT-4o和Qwen-2分别完成了34.7%和27.3%的有害请求。我们的研究结果凸显了针对网络智能体进行安全对齐程序的迫切需求。我们的基准测试可在此获取：https://safearena.github.io
英文摘要：
LLM-based agents are becoming increasingly proficient at solving web-based tasks. With this capability comes a greater risk of misuse for malicious purposes, such as posting misinformation in an online forum or selling illicit substances on a website. To evaluate these risks, we propose SafeArena, the first benchmark to focus on the deliberate misuse of web agents. SafeArena comprises 250 safe and 250 harmful tasks across four websites. We classify the harmful tasks into five harm categories -- misinformation, illegal activity, harassment, cybercrime, and social bias, designed to assess realistic misuses of web agents. We evaluate leading LLM-based web agents, including GPT-4o, Claude-3.5 Sonnet, Qwen-2-VL 72B, and Llama-3.2 90B, on our benchmark. To systematically assess their susceptibility to harmful tasks, we introduce the Agent Risk Assessment framework that categorizes agent behavior across four risk levels. We find agents are surprisingly compliant with malicious requests, with GPT-4o and Qwen-2 completing 34.7% and 27.3% of harmful requests, respectively. Our findings highlight the urgent need for safety alignment procedures for web agents. Our benchmark is available here: https://safearena.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
