arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03491cs.CVcs.CL

解耦骨架与血肉:基于解缠对齐和结构感知引导的高效多模态表格推理

Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance

Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳));Peng Cheng Laboratory, Shenzhen, China(鹏城实验室)

Yingjie Zhu, Xuefeng Bai, Kehai Chen, Yang Xiang, Youcheng Pan, Xiaoqiang Zhou, Min Zhang

AI总结:

提出DiSCo解缠结构-内容对齐框架和Table-GLS全局到局部结构引导推理框架,高效增强LVLM的表格理解与推理能力,无需昂贵监督或外部工具。

摘要:

由于复杂的布局和紧密耦合的结构-内容信息,对表格图像进行推理对于大型视觉语言模型(LVLM)仍然具有挑战性。现有解决方案通常依赖于昂贵的监督训练、强化学习或外部工具,限制了效率和可扩展性。这项工作解决了一个关键问题:如何以最少的标注且无需外部工具来使LVLM适应表格推理?具体来说,我们首先引入了DiSCo,一种解缠结构-内容对齐框架,在多模态对齐期间明确分离结构抽象和语义基础,高效地将LVLM适应于表格结构。在DiSCo的基础上,我们进一步提出了Table-GLS,一种全局到局部结构引导推理框架,通过结构化探索和基于证据的推理来执行表格推理。跨多个基准的大量实验表明,我们的框架高效地增强了LVLM的表格理解和推理能力,特别是泛化到未见过的表格结构。我们的数据和代码可在https://github.com/AAAndy-Zhu/TableVLM获取。

英文摘要:

Reasoning over table images remains challenging for Large Vision-Language Models (LVLMs) due to complex layouts and tightly coupled structure-content information. Existing solutions often depend on expensive supervised training, reinforcement learning, or external tools, limiting efficiency and scalability. This work addresses a key question: how to adapt LVLMs to table reasoning with minimal annotation and no external tools? Specifically, we first introduce DiSCo, a Disentangled Structure-Content alignment framework that explicitly separates structural abstraction from semantic grounding during multimodal alignment, efficiently adapting LVLMs to tables structures. Building on DiSCo, we further present Table-GLS, a Global-to-Local Structure-guided reasoning framework that performs table reasoning via structured exploration and evidence-grounded inference. Extensive experiments across diverse benchmarks demonstrate that our framework efficiently enhances LVLM's table understanding and reasoning capabilities, particularly generalizing to unseen table structures. Our data and code are available at https://github.com/AAAndy-Zhu/TableVLM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑