GraphVLM：多模态图学习的视觉语言模型基准测试
GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning
AI总结：
GraphVLM通过三种范式评估视觉语言模型在多模态图学习中的能力，发现VLM-as-Predictor在性能上表现最佳，展示了其在多模态图学习中的潜力。
摘要：
视觉语言模型（VLMs）在对齐和理解多模态信号方面表现出色，但其在结构化数据上的推理能力仍未被充分探索。为填补这一空白，我们提出了GraphVLM，一个系统性基准，用于评估和利用VLMs在多模态图学习（MMGL）中的能力。GraphVLM探讨了三种整合VLMs与图推理的互补范式：（1）VLM-as-Encoder，通过多模态特征融合增强图神经网络；（2）VLM-as-Aligner，通过潜在空间或语言空间桥梁促进基于LLM的结构化推理；（3）VLM-as-Predictor，直接将VLMs作为多模态主干用于图学习任务。在六个跨领域数据集上的广泛实验表明，VLMs通过这三种角色增强了多模态图学习。其中，VLM-as-Predictor实现了最显著且一致的性能提升，揭示了视觉语言模型在多模态图学习中的未被开发潜力。该基准代码在https://github.com/oamyjin/GraphVLM上公开。
英文摘要：
Vision-Language Models (VLMs) have demonstrated remarkable capabilities in aligning and understanding multimodal signals, yet their potential to reason over structured data, where multimodal entities are connected through explicit relational graphs, remains largely underexplored. Unlocking this capability is crucial for real-world applications such as social networks, recommendation systems, and scientific discovery, where multimodal information is inherently structured. To bridge this gap, we present GraphVLM, a systematic benchmark designed to evaluate and harness the capabilities of VLMs for multimodal graph learning (MMGL). GraphVLM investigates three complementary paradigms for integrating VLMs with graph reasoning: (1) VLM-as-Encoder, which enriches graph neural networks through multimodal feature fusion; (2) VLM-as-Aligner, which bridges modalities in latent or linguistic space to facilitate LLM-based structured reasoning; and (3) VLM-as-Predictor, which directly employs VLMs as multimodal backbones for graph learning tasks. Extensive experiments across six datasets from diverse domains demonstrate that VLMs enhance multimodal graph learning via all three roles. Among these paradigms, VLM-as-Predictor achieves the most substantial and consistent performance gains, revealing the untapped potential of vision-language models as a new foundation for multimodal graph learning. The benchmark code is publicly available at https://github.com/oamyjin/GraphVLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
