具身基准构建的智能自动化：流程、具身、模拟器与趋势
Intelligent Automation for Embodied Benchmark Construction: Pipelines, Embodiments, Simulators, and Trends
AI总结：
本文综述具身智能基准构建的五阶段流程，分析从人工到自动化再到智能体闭环的转变，指出自动化将成本转向验证与治理。
摘要：
具身智能现已涵盖导航、家务辅助、操作、自动驾驶、空中智能体及多模态大模型控制。这一扩展使得基准构建成为可靠评估的核心瓶颈。与静态数据集不同，具身基准将任务规范、环境、机器人数据、演示、标注、指标、评估脚本和发布策略整合为一个评估系统。本综述通过五阶段构建流程回顾文献：需求与任务构建、数据获取、数据清洗与标注、基准套件生成与指标定义、评估执行与诊断反馈。针对每个阶段，分析从人工管理到传统自动化、基础模型辅助以及智能体闭环工作流的转变。同时比较了人工、数据与资产获取、计算与仿真、验证与调试、治理与维护以及返工风险等定性构建成本。主要结论是：自动化并非简单降低基准成本，而是往往将成本转向验证、可审计性、版本控制和长期治理。因此，具身评估的进展不仅取决于更大的基准套件，还取决于可诊断、可审计且可负责任地更新的构建流程。
英文摘要：
Embodied intelligence now spans navigation, household assistance, manipulation, autonomous driving, aerial agents, and multimodal large-model control. This expansion has made benchmark construction a central bottleneck for reliable evaluation. Unlike static datasets, embodied benchmarks combine task specifications, environments, robot data, demonstrations, annotations, metrics, evaluation scripts, and release policies into a single evaluation system. This survey reviews the literature through a five-stage construction pipeline: requirement and task construction, data acquisition, data cleaning and annotation, benchmark suite generation and metric definition, and evaluation execution with diagnostic feedback. For each stage, the survey analyzes the transition from manual curation to traditional automation, foundation-model assistance, and agentic closed-loop workflows. It also compares qualitative construction costs across human labor, data and asset acquisition, compute and simulation, validation and debugging, governance and maintenance, and rework risk. The main conclusion is that automation does not simply reduce benchmark cost. Instead, it often shifts cost toward validation, auditability, version control, and long-term governance. Progress in embodied evaluation will therefore depend not only on larger benchmark suites, but also on construction pipelines that are diagnosable, auditable, and responsibly refreshable.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
