基于表格的思考：通过神经符号推理增强多模态表格理解
Thinking with Tables: Enhancing Multi-Modal Tabular Understanding via Neuro-Symbolic Reasoning
AI总结：
本文提出TWT框架，通过神经符号推理解决表格数据结构多样性、特征依赖复杂性及任务异质性问题，在八个数据集上验证了其在多模态表格理解任务中的优越性能。
摘要：
多模态大语言模型（MLLMs）在图像和文本等模态上展示了出色的推理能力。然而，表格数据作为关键现实世界模态，仍缺乏多模态学习的充分探索。本文聚焦于表格-视觉多模态理解（TVMU）任务，识别了三个核心挑战：（1）表格的高结构性差异和数据不完整性，（2）隐含且复杂的特征依赖关系，（3）下游任务中问题解决管道的显著异质性。为解决这些问题，我们提出了Thinking with Tables（TWT）。TWT采用程序辅助的代码基神经符号推理机制，通过与外部环境交互，促进关键操作如信息提取和元素建模。我们在八个代表性数据集上评估了TWT。实验结果表明，TWT在准确性上平均比现有基线高出10%，在TVMU任务中达到与甚至超越专有商业SOTA LLMs的性能。模型和代码可在https://github.com/kunyang-YU/Thinking-with-Tables获取。
英文摘要：
Multimodal Large Language Models (MLLMs) have demonstrated remarkable reasoning capabilities across modalities such as images and text. However, tabular data, despite being a critical real-world modality, remains relatively underexplored in multimodal learning. In this paper, we focus on the task of Tabular-Vision Multi-Modal Understanding (TVMU) and identify three core challenges: (1) high structural variability and data incompleteness in tables, (2) implicit and complex feature dependencies, and (3) significant heterogeneity in problem-solving pipelines across downstream tasks. To address these issues, we propose Thinking with Tables (TWT). TWT employs a program-aided code-based neuro-symbolic reasoning mechanism that facilitates key operations, such as information extraction and element modeling, by interacting with external environments. We evaluate TWT on eight representative datasets. Experimental results demonstrate that TWT consistently outperforms existing baselines by an average of 10\% in accuracy, achieving performance comparable to, or even surpassing, proprietary commercial SOTA LLMs on TVMU tasks. Models and codes are available at https://github.com/kunyang-YU/Thinking-with-Tables
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
