arXiv 2604.002702026-06-05cs.CV

OmniSch：面向结构化图表视觉推理的多模态PCB原理图基准

OmniSch: A Multimodal PCB Schematic Benchmark For Structured Diagram Visual Reasoning

Pennsylvania State University, USA（宾夕法尼亚州立大学）；Independent Researcher（独立研究者）；Binghamton University, USA（布ingham顿大学）；Shanghai Jiao Tong University, China（上海交通大学）；Microsoft Research（微软研究院）

Taiting Lu, Kaiyuan Lin, Yuxin Tian, Mingjia Wang, Yubo Wang, Muchuan Wang, Sharique Khatri, Akshit Kartik, Yixi Wang, Amey Santosh Rane, Yida Wang, Sung-Liang … 展开作者

Taiting Lu, Kaiyuan Lin, Yuxin Tian, Mingjia Wang, Yubo Wang, Muchuan Wang, Sharique Khatri, Akshit Kartik, Yixi Wang, Amey Santosh Rane, Yida Wang, Sung-Liang Chen, Yifan Yang, Yi-Chao Chen, Yincheng Jin, Mahanth Gowda

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出首个多模态PCB原理图理解基准OmniSch，包含四项任务评估大模型在视觉定位、图推理和几何推理上的能力，揭示现有模型在工程图表理解上的显著差距。

摘要：

近期大型多模态模型（LMMs）在视觉定位、文档理解和图表推理任务中取得了快速进展。然而，它们将印刷电路板（PCB）原理图转换为机器可读的空间加权网表图（同时捕获组件属性、连接性和几何信息）的能力仍未被充分探索，尽管这种图表示是实际电子设计自动化（EDA）工作流的基石。为弥补这一差距，我们引入了OmniSch，这是首个旨在评估LMMs在原理图理解和空间网表图构建方面的综合基准。OmniSch包含1,854张真实世界原理图，并包括四项任务：（1）原理图实体的视觉定位，包含109.9K个定位实例，将423.4K个图表语义标签与其视觉区域对齐；（2）图到图推理，理解图表元素间的拓扑关系；（3）几何推理，为每个连接构建依赖于布局的权重；（4）用于视觉搜索的工具增强型智能体推理，调用外部工具完成（1）-（3）。我们的结果揭示了当前LMMs在解释原理图工程制品方面的显著差距，包括不可靠的细粒度定位、脆弱的布局到图解析、不一致的全局连通性推理以及低效的视觉探索。

英文摘要：

Recent large multimodal models (LMMs) have made rapid progress in visual grounding, document understanding, and diagram reasoning tasks. However, their ability to convert Printed Circuit Board (PCB) schematic diagrams into machine-readable spatially weighted netlist graphs, jointly capturing component attributes, connectivity, and geometry, remains largely underexplored, despite such graph representations are the backbone of practical electronic design automation (EDA) workflows. To bridge this gap, we introduce OmniSch, the first comprehensive benchmark designed to assess LMMs on schematic understanding and spatial netlist graph construction. OmniSch contains 1,854 real-world schematic diagrams and includes four tasks: (1) visual grounding for schematic entities, with 109.9K grounded instances aligning 423.4K diagram semantic labels to their visual regions; (2) diagram-to-graph reasoning, understanding topological relationship among diagram elements; (3) geometric reasoning, constructing layout-dependent weights for each connection; and (4) tool-augmented agentic reasoning for visual search, invoking external tools to accomplish (1)-(3). Our results reveal substantial gaps of current LMMs in interpreting schematic engineering artifacts, including unreliable fine-grained grounding, brittle layout-to-graph parsing, inconsistent global connectivity reasoning and inefficient visual exploration.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。