视觉大模型 / VLM

Zero-Shot Document Understanding using Pseudo Table of Contents-Guided Retrieval-Augmented Generation

Hyeon Seong Jeong, Sangwoo Jo, Byeong Hyun Yoon, Yoonseok Heo, Haedong Jeong, Taehoon Kim

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Does visualization help AI understand data?

Victoria R. Li, Johnathan Sun, Martin Wattenberg

机构 * Harvard University（哈佛大学） ； Google Research（谷歌研究） 纠错

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments 5 pages, 6 figures

In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understanding

Wan-Cyuan Fan, Yen-Chun Chen, Mengchen Liu, Alexander Jacobson, Lu Yuan, Leonid Sigal

机构 * UBC（不列颠哥伦比亚大学） ； Microsoft（微软） ； Vector Institute for AI（人工智能向量研究所） ； CIFAR AI Chair（卡尔·弗雷德里克人工智能主席） 纠错

专题命中 文档图表理解 ：vision language model(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2407.14506

Seeing the Signs: A Survey of Edge-Deployable OCR Models for Billboard Visibility Analysis

Maciej Szankin, Vidhyananth Venkatasamy, Lihang Ying

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.AI

PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Tricks

Feng Ni, Kui Huang, Yao Lu, Wenyu Lv, Guanzhong Wang, Zeyu Chen, Yi Liu

机构 * PaddlePaddle Team, Baidu Inc.（百度公司） 纠错

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Understanding Generative AI Capabilities in Everyday Image Editing Tasks

Mohammad Reza Taesiri, Brandon Collins, Logan Bolton, Viet Dac Lai, Franck Dernoncourt, Trung Bui, Anh Totti Nguyen

专题命中 文档图表理解 ：VLM(abstract)；分类 cs.CV、cs.AI

Comments Code and qualitative examples are available at: https://psrdataset.github.io

Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval

Alexander Most, Joseph Winjum, Ayan Biswas, Shawn Jones, Nishath Rajiv Ranasinghe, Dan O'Malley, Manish Bhattarai

机构 * Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室） ； Gianforte School of Computing（吉安福特计算学院） ； Montana State University Bozeman（蒙大拿州立大学博兹曼分校） 纠错

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.CV、cs.AI

FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding

De-An Huang, Subhashree Radhakrishnan, Zhiding Yu, Jan Kautz

机构 * NVIDIA（英伟达） 纠错

专题命中 文档图表理解 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Towards a Multimodal Document-grounded Conversational AI System for Education

Karan Taneja, Anjali Singh, Ashok K. Goel

机构 * Georgia Institute of Technology（佐治亚理工学院） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.CV、cs.AI

Comments 15 pages, 4 figures, AIED 2025

MathWriting: A Dataset For Handwritten Mathematical Expression Recognition

Philippe Gervais, Anastasiia Fadeeva, Andrii Maksai

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.LG

OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations

Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, Jin Shi, Fan Wu, Pei Chu, Minghao Liu, Zhenxiang Li, Chao Xu, Bo Zhang, Botian Shi, Zhongying Tu, Conghui He

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR2025

ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understanding

Zhengzhuo Xu, Bowen Qu, Yiyan Qi, Sinan Du, Chengjin Xu, Chun Yuan, Jian Guo

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs

Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Yi Su, Bohua Chen, Dongping Chen, Siyuan Wu, Xing Zhou, Wenbin Jiang, Hai Jin, Xiangliang Zhang

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments WWW'25

MFTF: Mask-free Training-free Object Level Layout Control Diffusion Model

Shan Yang

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 7 figures

M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding

Jaemin Cho, Debanjan Mahata, Ozan Irsoy, Yujie He, Mohit Bansal

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Project webpage: https://m3docrag.github.io

TableGPT2: A Large Multimodal Model with Tabular Data Integration

Aofeng Su, Aowen Wang, Chao Ye, Chen Zhou, Ga Zhang, Gang Chen, Guangcheng Zhu, Haobo Wang, Haokai Xu, Hao Chen, Haoze Li, Haoxuan Lan, Jiaming Tian, Jing Yuan, Junbo Zhao, Junlin Zhou, Kaizhe Shou, Liangyu Zha, Lin Long, Liyao Li, Pengzuo Wu, Qi Zhang, Qingyi Huang, Saisai Yang, Tao Zhang, Wentao Ye, Wufang Zhu, Xiaomeng Hu, Xijun Gu, Xinjie Sun, Xiang Li, Yuhang Yang, Zhiqing Xiao

专题命中 文档图表理解 ：visual language model(abstract)；分类 cs.AI、cs.LG

OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text

Qingyun Li, Zhe Chen, Weiyun Wang, Wenhai Wang, Shenglong Ye, Zhenjiang Jin, Guanzhou Chen, Yinan He, Zhangwei Gao, Erfei Cui, Jiashuo Yu, Hao Tian, Jiasheng Zhou, Chao Xu, Bin Wang, Xingjian Wei, Wei Li, Wenjian Zhang, Bo Zhang, Pinlong Cai, Licheng Wen, Xiangchao Yan, Zhenxiang Li, Pei Chu, Yi Wang, Min Dou, Changyao Tian, Xizhou Zhu, Lewei Lu, Yushi Chen, Junjun He, Zhongying Tu, Tong Lu, Yali Wang, Limin Wang, Dahua Lin, Yu Qiao, Botian Shi, Conghui He, Jifeng Dai

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

TRINS: Towards Multimodal Language Models that Can Read

Ruiyi Zhang, Yanzhe Zhang, Jian Chen, Yufan Zhou, Jiuxiang Gu, Changyou Chen, Tong Sun

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024

TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document

Yuliang Liu, Biao Yang, Qiang Liu, Zhang Li, Zhiyin Ma, Shuo Zhang, Xiang Bai

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.CV、cs.AI

DUBLIN -- Document Understanding By Language-Image Network

Kriti Aggarwal, Aditi Khandelwal, Kumar Tanmay, Owais Mohammed Khan, Qiang Liu, Monojit Choudhury, Hardik Hansrajbhai Chauhan, Subhojit Som, Vishrav Chaudhary, Saurabh Tiwary

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Enhancing Document Information Analysis with Multi-Task Pre-training: A Robust Approach for Information Extraction in Visually-Rich Documents

Tofik Ali, Partha Pratim Roy

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Mariya Hendriksen, Svitlana Vakulenko, Ernst Kuiper, Maarten de Rijke

专题命中 文档图表理解 ：VLM(abstract)；分类 cs.CV、cs.LG

Comments 18 pages, accepted as a reproducibility paper at ECIR 2023

Towards Complex Document Understanding By Discrete Reasoning

Fengbin Zhu, Wenqiang Lei, Fuli Feng, Chao Wang, Haozhou Zhang, Tat-Seng Chua

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments ACM MM 2022

ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules

Zhi-Qi Cheng, Qi Dai, Siyao Li, Jingdong Sun, Teruko Mitamura, Alexander G. Hauptmann

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Hierarchical multimodal transformers for Multi-Page DocVQA

Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

Nico Daheim, David Thulke, Christian Dugast, Hermann Ney

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Accepted to Findings of EMNLP 2022

Cascaded Span Extraction and Response Generation for Document-Grounded Dialog

Nico Daheim, David Thulke, Christian Dugast, Hermann Ney

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Accepted by 1st DialDoc Workshop at ACL-IJCNLP 2021

DocClaw: A Unified Agentic System for Intelligent Document Processing

DocClaw：用于智能文档处理的统一智能体系统

Siqi Xiang, Zhipeng Xu, Yufei Liu, Junhao Ji, Qing Liu, Zulong Chen, Zhibo Yang, Chunyan Miao, Shijian Lu

专题命中 文档图表理解 ：VLM(abstract_cn)；分类 cs.CV

AI总结 研究针对现有智能文档处理任务多为独立建模的问题，提出统一智能体系统DocClaw，通过智能体与文档的共享交互处理多种IDP任务，在基准测试中取得有竞争力的性能。

Code as Representation: A Compilable Parsing Paradigm for Academic Documents

代码作为表示：学术文档的可编译解析范式

Rihui Jin, Jun Wang, chengyuan zhu, Liang Mingyu, Yue Gao, Li Yunxuan, Kuicai Dong, Guilin Qi, Lin Ren, Yongrui Chen, Xinbang Dai, Jiaqi Li, Tongtong Wu, Gholamreza Haffari

机构 * Southeast University（东南大学） ； Nanyang Technological University（南洋理工大学） ； Nanjing University（南京大学） 纠错

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV

AI总结 针对学术PDF难以被机器处理的问题，提出CADP可编译解析范式，构建CADP-Bench基准并测试SOTA MLLMs，发现前沿模型仍难生成高保真可执行重构，该基准已开放供研究。

Comments Accepted by ACM MM 2026

ConceptFormer: Learning Adaptive Latent Concepts for Query-Document Alignment in Visual Document Retrieval

ConceptFormer：学习自适应潜在概念以实现视觉文档检索中的查询-文档对齐

Peng Chunyi, Xu Zhipeng, Yan Yukun, Liu Zhenghao, Yu Shi, Mei Sen, Sun Yubo, Zhang Yongheng, Zhou Jie, Gu Yu, Yu Ge, Sun Maosong

机构 * Northeastern University（东北大学） ； Tsinghua University（清华大学） ； Peking University（北京大学） 纠错

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV

AI总结 针对视觉文档检索中现有监督信号的局限，本文提出ConceptFormer框架，以自适应潜在概念为中间表示衔接语义鸿沟，在基准测试中较最强基线实现了16.7%、22.1%的NDCG@10相对提升，性能优异。