arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.20146cs.AI
收藏

BIM-Edit:基于IFC的建筑信息模型的大语言模型基准测试

BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling

University of Rostock(罗斯托克大学);Clausthal University of Technology(克劳斯塔尔工业大学)

Bharathi Kannan Nithyanantham, Clemens Kujat, Tobias Sesterhenn, Stefan Telgmann, Ashwin Nedungadi, Jörn Plönnigs, Christian Bartelt, Stefan Lüdtke

AI总结:

提出BIM-Edit基准,评估大语言模型在IFC格式建筑信息模型上的自然语言编辑能力,涵盖324个任务,最佳模型平均得分仅49.5%,揭示当前能力与工程需求间的差距。

摘要:

大语言模型(LLMs)越来越多地被应用于计算机辅助设计(CAD),以从文本指令生成设计工件。在工程实践中,这需要的不仅仅是创建新的几何体,模型还必须理解现有场景,正确编辑它们,并保留语义和关系。然而,许多CAD基准侧重于创建新模型而非编辑现有模型,并且主要评估几何正确性。我们引入了BIM-Edit,这是一个用于评估LLMs在行业基础类(IFC)格式表示的建筑信息模型(BIM)上进行自然语言编辑的基准。BIM提供了一个具有挑战性的测试平台,因为建筑模型将几何体与语义和关系结构编码在一起。BIM-Edit包含324个编辑任务,涵盖11个真实建筑模型和36个合成场景。任务使用三种指令类别——直接、空间和拓扑——表达,涵盖显式编辑和场景接地编辑。我们沿三个维度评估输出:几何准确性、语义有效性和拓扑一致性。在评估的LLMs中,表现最佳的模型在三个指标上的平均得分仅为49.5%,且没有模型完全解决超过3.4%的任务。这些结果表明当前LLM能力与结构化工程设计工作流的要求之间存在巨大差距。

英文摘要:

Large language models (LLMs) are increasingly applied to computer-aided design (CAD) to generate design artifacts from textual instructions. In engineering practice, this requires more than creating new geometry, models must also understand existing scenes, edit them correctly, and preserve semantics and relations. However, many CAD benchmarks focus on creating new models rather than editing existing ones, and mostly evaluate geometric correctness. We introduce BIM-Edit, a benchmark for evaluating LLMs on natural-language editing of Building Information Models (BIM) represented in the Industry Foundation Classes (IFC) format. BIM provides a challenging testbed because building models encode geometry together with semantic and relational structure. BIM-Edit contains 324 editing tasks spanning 11 realistic building models and 36 synthetic scenes. Tasks are expressed using three instruction categories - direct, spatial, and topological - covering both explicit and scene-grounded edits. We evaluate outputs along three dimensions: geometric accuracy, semantic validity, and topological consistency. Across evaluated LLMs, the best-performing model achieves only 49.5% average score across the three metrics, and no model fully solves more than 3.4% of tasks. These results demonstrate a substantial gap between current LLM capabilities and the requirements of structured engineering design workflows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑