静态化与即插即用:让具身智能评估更简单
Static and Plugged: Make Embodied Evaluation Simple
AI总结:
针对具身智能评估成本高、难扩展的问题,提出StaticEmbodiedBench基准,利用静态场景表征实现即插即用的统一评估,涵盖42场景与8维度,评估30个模型并发布200样本子集。
摘要:
具身智能正快速发展,亟需高效评估。现有基准多依赖交互式仿真环境或真实世界设置,成本高、碎片化且难以扩展。为此,我们提出StaticEmbodiedBench,一个即插即用的基准,通过静态场景表征实现统一评估。它涵盖42种多样场景和8个核心维度,支持通过简单接口进行可扩展且全面的评估。此外,我们评估了19个视觉语言模型(VLM)和11个视觉语言动作模型(VLA),建立了首个具身智能统一静态排行榜。同时,我们发布基准中200个样本的子集,以加速具身智能发展。
英文摘要:
Embodied intelligence is advancing rapidly, driving the need for efficient evaluation. Current benchmarks typically rely on interactive simulated environments or real-world setups, which are costly, fragmented, and hard to scale. To address this, we introduce StaticEmbodiedBench, a plug-and-play benchmark that enables unified evaluation using static scene representations. Covering 42 diverse scenarios and 8 core dimensions, it supports scalable and comprehensive assessment through a simple interface. Furthermore, we evaluate 19 Vision-Language Models (VLMs) and 11 Vision-Language-Action models (VLAs), establishing the first unified static leaderboard for Embodied intelligence. Moreover, we release a subset of 200 samples from our benchmark to accelerate the development of embodied intelligence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。