发表机构
Tsinghua University; Zhiman Inc.; Chongqing University; UCAS; Shandong University; BUPT; Fudan University; Henan Polytechnic University; Peking University; Zhejiang University（清华大学; 知满科技; 重庆大学; 中国科学院大学; 山东大学; 北京邮电大学; 复旦大学; 河南理工大学; 北京大学; 浙江大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
EngiWorld是首个围绕完整设计循环的工程基准，含1,301个任务和6个领域，评估显示最强模型仅得44.3分，揭示了前沿智能体在专业工程自动化上的巨大能力差距。
AI 中文摘要
自主智能体在通用计算机使用方面取得了快速进展，但专业工业工程的可靠自动化仍遥不可及，因为工程工作流要求对几何和物理约束及依赖关系进行推理，这些约束和依赖关系在软件和设计阶段中得以保留。我们提出了EngiWorld，这是第一个围绕完整设计循环构建的基准：包含1,301个专家精选任务，涵盖6个工程领域（CAD、CAE、CAM、BIM、EDA和3D可视化）和26个专业软件平台，同时提供GUI和CLI接口，以及从软件选择到开放式任务的6种任务类型。我们进一步引入了一种以工件为中心的评估方法，该方法基于统一的领域验证器套件，以编程方式检查最终和中间工件的几何有效性、物理可行性和规则合规性，并根据规范达成度连续评分定量设计任务，而非采用二元成功标准。对七个前沿模型的评估揭示了显著的能力差距：最强的模型仅达到44.3的EngiScore，且仅3.6%的多软件尝试成功。EngiWorld为衡量智能体端到端操作专业工程软件进展提供了首个严格基础。
英文摘要
Autonomous agents have made rapid progress in general-purpose computer use, but reliable automation of professional industrial engineering remains out of reach, as engineering workflows demand reasoning over geometric and physical constraints and dependencies preserved across software and design stages. We present EngiWorld, the first benchmark structured around the complete design loop: 1,301 expert-curated tasks spanning 6 engineering domains (CAD, CAE, CAM, BIM, EDA, and 3D visualization) and 26 professional software platforms, with both GUI and CLI interfaces and 6 task types ranging from software-selection to open-ended tasks. We further introduce an artifact-centric evaluation methodology built on a unified domain-verifier suite, which programmatically checks the geometric validity, physical feasibility, and rule compliance of final and intermediate artifacts, and scores quantitative design tasks continuously by specification attainment rather than binary success. Evaluation of seven frontier models reveals a substantial capability gap: the strongest model achieves an EngiScore of only 44.3, and just 3.6% of multi-software attempts succeed. EngiWorld provides the first rigorous foundation for measuring progress toward agents that operate professional engineering software end to end.
CommentsProject page: https://engiworld.github.io
