arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.03478cs.HC
收藏

用于增强大型多模态模型可视化理解的 SVG 分解:以平面图为例的研究

SVG Decomposition for Enhancing Large Multimodal Models Visualization Comprehension: A Study with Floor Plans

University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);City St. George's University London(伦敦城市圣乔治大学)

Jeongah Lee, Ali Sarvghad

AI总结:

本研究探讨将SVG作为分解策略以提升大型多模态模型对平面图的理解,发现SVG与光栅输入结合能提升空间理解但会阻碍寻路等空间推理,揭示了分解策略的前景与局限。

摘要:

大型多模态模型(LMMs)解释可视化的能力日益增强,但在空间推理方面仍存在困难。一种提出的策略是分解,即将复杂的可视化分解为结构化组件。在本研究中,我们探讨了将可缩放矢量图形(SVGs)作为一种分解策略,以提升 LMMs 在平面图理解任务上的表现的有效性。平面图是一个有价值的测试平台,因为它们结合了几何、拓扑和语义,且其可靠理解具有现实应用,例如盲人和低视力个体的无障碍访问。我们使用三个 LMMs(GPT-4o、Claude 3.7 Sonnet 和 Llama 3.2 11B Vision Instruct)对 75 个平面图进行了探索性研究。结果表明,将 SVG 与光栅输入结合(SVG+PNG)能提升在空间理解任务上的表现,但往往会阻碍空间推理,尤其是在寻路方面。这些发现突显了分解作为推进空间可视化理解策略的前景与局限性。

英文摘要:

Large multimodal models (LMMs) are increasingly capable of interpreting visualizations, yet they continue to struggle with spatial reasoning. One proposed strategy is decomposition, which breaks down complex visualizations into structured components. In this work, we examine the efficacy of scalable vector graphics (SVGs) as a decomposition strategy for improving LMMs' performance on floor plans comprehension. Floor plans serve as a valuable testbed because they combine geometry, topology, and semantics, and their reliable comprehension has real-world applications, such as accessibility for blind and low-vision individuals. We conducted an exploratory study with three LMMs (GPT-4o, Claude 3.7 Sonnet, and Llama 3.2 11B Vision Instruct) across 75 floor plans. Results show that combining SVG with raster input (SVG+PNG) improves performance on spatial understanding tasks but often hinders spatial reasoning, particularly in pathfinding. These findings highlight both the promise and limitations of decomposition as a strategy for advancing spatial visualization comprehension.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑