DynamicGTR: 利用图拓扑表示偏好提升视觉语言模型在图问答中的能力
DynamicGTR: Leveraging Graph Topology Representation Preferences to Boost VLM Capabilities on Graph QAs
AI总结:
DynamicGTR通过动态选择最优图拓扑表示,提升视觉语言模型在图问答中的零样本性能及跨任务迁移能力。
摘要:
视觉语言模型(VLMs)已发展为在各种领域中进行零样本问答(QA)的多功能解决方案。然而,使VLMs有效理解结构化图并执行准确且高效的问答仍具挑战性。现有方法通常依赖单一图拓扑表示(GTR),如固定风格的视觉图像或统一的文本描述。这种“一刀切”策略往往忽视了模型特定和任务特定的偏好,导致对图相关查询的响应不准确或过长。为此,我们提出了DynamicGTR框架,该框架在推理过程中动态选择最优的GTR,从而通过可定制的准确性和简洁性权衡提升VLMs的零样本图问答能力。大量实验表明,DynamicGTR不仅提高了基于VLM的图算法问答性能,还成功将从合成图算法任务中训练的经验转移到现实应用中,如链接预测和节点分类,而无需额外训练。此外,DynamicGTR在任务、领域和模型之间表现出强大的迁移能力,表明其作为广泛图场景灵活解决方案的潜力。
英文摘要:
Vision-Language Models (VLMs) have emerged as versatile solutions for zero-shot question answering (QA) across various domains. However, enabling VLMs to effectively comprehend structured graphs and perform accurate, efficient QA remains challenging. Existing approaches typically rely on one single graph topology representation (GTR), such as fixed-style visual images or unified text descriptions. This ``one-size-fits-all'' strategy often neglects model-specific and task-specific preferences, resulting in inaccurate or over-lengthy responses to graph-related queries. To address this, we propose the $\mbox{DynamicGTR}$ framework, which dynamically selects the optimal GTR for each query during inference, thereby enhancing the zero-shot graph QA capabilities of VLMs with a customizable accuracy and brevity trade-off. Extensive experiments show that DynamicGTR not only improves VLM-based graph algorithm QA performance but also successfully transfers the experience trained from synthetic graph algorithm tasks to real-world applications like link prediction and node classification, without any additional training. Additionally, DynamicGTR demonstrates strong transferability across tasks, domains, and models, suggesting its potential as a flexible solution for broad graph scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。