AEC-Bench：一种用于建筑、工程和施工领域代理系统的多模态基准
AEC-Bench: A Multimodal Benchmark for Agentic Systems in Architecture, Engineering, and Construction
AI总结：
AEC-Bench旨在评估建筑、工程和施工领域代理系统在真实任务中的表现，涵盖图纸理解、跨表单推理和项目级协调任务，通过多模型基准测试提升基础模型性能。
摘要：
AEC-Bench是一种多模态基准，用于评估建筑、工程和施工（AEC）领域中代理系统在真实任务中的性能。该基准涵盖需要图纸理解、跨表单推理和施工项目级协调的任务。本报告描述了基准的动机、数据集分类、评估协议以及在多个领域特定基础模型中的基线结果。我们使用AEC-Bench来识别一致的工具和基础模型设计技术，以在基础模型自身基础中统一提升性能，例如Claude Code和Codex。我们公开发布了我们的基准数据集、代理基础模型和评估代码，以确保完全可复现性，地址为https://github.com/nomic-ai/aec-bench，采用Apache 2许可证。
英文摘要：
The AEC-Bench is a multimodal benchmark for evaluating agentic systems on real-world tasks in the Architecture, Engineering, and Construction (AEC) domain. The benchmark covers tasks requiring drawing understanding, cross-sheet reasoning, and construction project-level coordination. This report describes the benchmark motivation, dataset taxonomy, evaluation protocol, and baseline results across several domain-specific foundation model harnesses. We use AEC-Bench to identify consistent tools and harness design techniques that uniformly improve performance across foundation models in their own base harnesses, such as Claude Code and Codex. We openly release our benchmark dataset, agent harness, and evaluation code for full replicability at https://github.com/nomic-ai/aec-bench under an Apache 2 license.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
