EARBench：面向基于基础模型的具身AI智能体任务规划的物理风险意识评估
EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents
AI总结：
针对基于基础模型的具身AI智能体任务规划缺乏物理风险意识的问题，提出EARBench自动化评估框架，通过多智能体协作评估风险，发现现有模型平均任务风险率达95.75%，为安全部署提供评估工具与研究方向。
摘要：
具身人工智能（EAI）将先进AI模型集成到物理实体中以实现现实世界交互。基础模型作为EAI智能体的“大脑”用于高级任务规划，已展现出良好前景。然而，这些智能体在物理环境中的部署存在重大安全挑战，例如缺乏足够风险意识的家政机器人可能将金属容器放入微波炉，潜在引发火灾。为解决这些关键安全问题，全面的部署前风险评估至关重要。本研究引入EARBench，一个用于EAI场景中自动化物理风险评估的新型框架。该框架采用多智能体协作系统，利用多种基础模型生成安全指南、创建高风险场景、制定任务规划并系统评估安全性。利用此框架，我们构建了EARDataset，包含跨多个领域的多样测试案例，涵盖文本和视觉场景。对最先进基础模型的全面评估显示出令人担忧的结果：所有模型均表现出高任务风险率（TRR），所有评估模型的平均TRR为95.75%。为应对这些挑战，我们进一步提出两种基于提示的风险缓解策略。尽管这些策略在降低TRR方面显示出一定效果，但改进有限，仍表明存在重大安全问题。本研究首次对EAI智能体的物理风险意识进行大规模评估，研究结果强调了增强EAI系统安全措施的迫切需求，并为开发更安全的具身人工智能系统提供了有价值的未来研究方向。数据和代码可在https://github.com/zihao-ai/EARBench获取。
英文摘要：
Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction. The emergence of foundation models as the "brain" of EAI agents for high-level task planning has shown promising results. However, the deployment of these agents in physical environments presents significant safety challenges. For instance, a housekeeping robot lacking sufficient risk awareness might place a metal container in a microwave, potentially causing a fire. To address these critical safety concerns, comprehensive pre-deployment risk assessments are imperative. This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios. EAIRiskBench employs a multi-agent cooperative system that leverages various foundation models to generate safety guidelines, create risk-prone scenarios, make task planning, and evaluate safety systematically. Utilizing this framework, we construct EARDataset, comprising diverse test cases across various domains, encompassing both textual and visual scenarios. Our comprehensive evaluation of state-of-the-art foundation models reveals alarming results: all models exhibit high task risk rates (TRR), with an average of 95.75% across all evaluated models. To address these challenges, we further propose two prompting-based risk mitigation strategies. While these strategies demonstrate some efficacy in reducing TRR, the improvements are limited, still indicating substantial safety concerns. This study provides the first large-scale assessment of physical risk awareness in EAI agents. Our findings underscore the critical need for enhanced safety measures in EAI systems and provide valuable insights for future research directions in developing safer embodied artificial intelligence system. Data and code are available at https://github.com/zihao-ai/EARBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
