Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existing multimodal RAG pipelines primarily face two critical challenges: first, standard semantic similarity retrievers frequently fetch topically overlapping yet answer-void distractor pages that mislead downstream generation; second, rigid single-pass pipelines heavily depend on initial retrieval success, where any omission of core evidence inevitably causes cascading errors. To address these challenges, we introduce HIEVI-RAG, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding. HIEVI-RAG systematically factorizes complex queries into a cooperative four-stage pipeline: (1) hierarchical question decomposition to break multi-hop root queries into atomic child questions; (2) coarse visual page retrieval leveraging a multimodal retriever to fetch candidate pages based on semantic similarity; (3) fine-grained page verification via EVIAGENT, a specialized multi-page verifier trained with GRPO to execute cross-page reasoning over multi-image blocks; and (4) memory-guided iterative generation that leverages accumulated sub-question context to execute multi-round, dynamic reasoning over the prioritized sequence. Extensive evaluations across four benchmarks demonstrate the robust efficacy and synergy of our framework, which significantly outperforms existing open-source baselines and exceeds the strongest reported baseline by an average of 8.05% in accuracy.
Visual Document Understanding and Reasoning: A Multi-Agent Collaboration Framework with Agent-Wise Adaptive Test-Time Scaling
National University of Singapore（新加坡国立大学）；Tencent Youtu Lab（腾讯优图实验室）；Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；Nanyang Technological University（南洋理工大学）；Zhejiang University（浙江大学）；DeepWisdom（深智科技）
Towards Complex Document Understanding By Discrete Reasoning
National University of Singapore（新加坡国立大学）；Estates Pte Ltd（6Estates私人有限公司）；Sichuan University（四川大学）；University of Science and Technology of China（中国科学技术大学）
Medical document understanding has gained much attention recently. One representative task is the International Classification of Disease (ICD) diagnosis code assignment. Existing work adopts either RNN or CNN as the backbone network because the vanilla BERT cannot handle well long documents (>2000 to kens). One issue shared across all these approaches is that they are over specific to the ICD code assignment task, losing generality to give the whole document-level and sentence-level embedding. As a result, it is not straight-forward to direct them to other downstream NLU tasks. Motivated by t…
KH-FUNSD: A Hierarchical and Fine-Grained Layout Analysis Dataset for Low-Resource Khmer Business Document
National Engineering Research Center of Speech and Language Information Processing (NERC-SLIP) University of Science and Technology of China（国家语信息处理语音工程研究中心（NERC-SLIP）大学科学技术大学）
MURE: Hierarchical Multi-Resolution Encoding via Vision-Language Models for Visual Document Retrieval
National University of Singapore, Singapore（新加坡国立大学）；University of Science and Technology of China, China（中国科学技术大学）；Hefei University of Technology, China（合肥工业大学）
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Wisconsin-Milwaukee（威斯康星大学密尔沃基分校）
This paper presents an efficient adaptation and application of the Dempster-Shafer theory of evidence, one that can be used effectively in a massively parallel hierarchical system for visual pattern perception. It describes the techniques used, and shows in an extended example how they serve to improve the system's performance as it applies a multiple-level set of processes.
Renmin University of China（中国人民大学）；University of Adelaide（阿德莱德大学）
Cross-modal retrieval between videos and texts has attracted growing attentions due to the rapid emergence of videos on the web. The current dominant approach for this problem is to learn a joint embedding space to measure cross-modal similarities. However, simple joint embeddings are insufficient to represent complicated visual and textual details, such as scenes, objects, actions and their compositions. To improve fine-grained video-text retrieval, we propose a Hierarchical Graph Reasoning (HGR) model, which decomposes video-text matching into global-to-local levels. To be specific, the mode…
Media and Data Science Research Labs, Adobe（Adobe 媒体与数据科学研究实验室）；Adobe Experience Cloud（Adobe 体验云）
Structure extraction from document images has been a long-standing research topic due to its high impact on a wide range of practical applications. In this paper, we share our findings on employing a hierarchical semantic segmentation network for this task of structure extraction. We propose a prior based deep hierarchical CNN network architecture that enables document structure extraction using very high resolution(1800 x 1000) images. We divide the document image into overlapping horizontal strips such that the network segments a strip and uses its prediction mask as prior for predicting the…
Translating renderings (e. g. PDFs, scans) into hierarchical document structures is extensively demanded in the daily routines of many real-world applications. However, a holistic, principled approach to inferring the complete hierarchical structure of documents is missing. As a remedy, we developed "DocParser": an end-to-end system for parsing the complete document structure - including all text elements, nested figures, tables, and table cell structures. Our second contribution is to provide a dataset for evaluating hierarchical document structure parsing. Our third contribution is to propos…
University of Science and Technology of China（中国科学技术大学）；iFLYTEK Research（科大讯飞研究院）
The problem of document structure reconstruction refers to converting digital or scanned documents into corresponding semantic structures. Most existing works mainly focus on splitting the boundary of each element in a single document page, neglecting the reconstruction of semantic structure in multi-page documents. This paper introduces hierarchical reconstruction of document structures as a novel task suitable for NLP and CV fields. To better evaluate the system performance on the new task, we built a large-scale dataset named HRDoc, which consists of 2,500 multi-page documents with nearly 2…
Department of Computer Science, University of Virginia（弗吉尼亚大学计算机科学系）；Department of System and Information Engineering, University of Virginia（弗吉尼亚大学系统与信息工程系）；Data Science Institute, University of Virginia（弗吉尼亚大学数据科学研究所）
The continually increasing number of documents produced each year necessitates ever improving information processing methods for searching, retrieving, and organizing text. Central to these information processing methods is document classification, which has become an important application for supervised learning. Recently the performance of these traditional classifiers has degraded as the number of documents has increased. This is because along with this growth in the number of documents has come an increase in the number of categories. This paper approaches this problem differently from cur…
DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Massachusetts Institute of Technology（麻省理工学院）；Stanford University（斯坦福大学）；Kookmin University（国民大学）；MIT-IBM Watson AI Lab, IBM Research（麻省理工学院-IBM沃森人工智能实验室，IBM研究院）
ORCA: Orchestrated Reasoning with Collaborative Agents for Document Visual Question Answering
Digital Research Center of Sfax, SMARTS Laboratory（突尼斯斯法克斯数字研究中心，SMARTS实验室）；École Polytechnique de Tunisie, University of Carthage（突尼斯理工学院，卡塔赫纳大学）；Computer Vision Center, Universitat Autònoma de Barcelona（巴塞罗那自治大学计算机视觉中心）
Incorporating relational reasoning in neural networks for object recognition remains an open problem. Although many attempts have been made for relational reasoning, they generally only consider a single type of relationship. For example, pixel relations through self-attention (e.g., non-local networks), scale relations through feature fusion (e.g., feature pyramid networks), or object relations through graph convolutions (e.g., reasoning-RCNN). Little attention has been given to more generalized frameworks that can reason across these relationships. In this paper, we propose a hierarchical re…
ADCD-Net: Robust Document Image Forgery Localization via Adaptive DCT Feature and Hierarchical Content Disentanglement
State Key Laboratory of Internet of Things for Smart City, Department of Computer and Information Science, University of Macau（澳门大学物联网智能城市国家重点实验室，计算机与信息科学系）；School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件工程学院）
Beijing National Research Center for Information Science and Technology (BNRist)（北京信息科学与技术国家研究中心）；Tsinghua University（清华大学）
In this paper, we investigate the problem of facial kinship verification by learning hierarchical reasoning graph networks. Conventional methods usually focus on learning discriminative features for each facial image of a paired sample and neglect how to fuse the obtained two facial image features and reason about the relations between them. To address this, we propose a Star-shaped Reasoning Graph Network (S-RGN). Our S-RGN first constructs a star-shaped graph where each surrounding node encodes the information of comparisons in a feature dimension and the central node is employed as the brid…
HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding
South China University of Technology（华南理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shenzhen Institute of Advanced Technology, CAS（中国科学院深圳先进技术研究院）；Nanjing University（南京大学）
Tongji University（同济大学）；DeepBlue Academy of Sciences（深兰科学院）；City University of Hong Kong（香港城市大学）
Reasoning-based approaches have demonstrated their powerful ability for the task of image-text matching. In this work, two issues are addressed for image-text matching. First, for reasoning processing, conventional approaches have no ability to find and use multi-level hierarchical similarity information. To solve this problem, a hierarchical similarity reasoning module is proposed to automatically extract context information, which is then co-exploited with local interaction information for efficient reasoning. Second, previous approaches only consider learning single-stream similarity alignm…
Federal University of Technology - Parana（巴拉那联邦理工大学）；University of Maryland, College Park（马里兰大学帕克分校）
Reasoning about images/objects and their hierarchical interactions is a key concept for the next generation of computer vision approaches. Here we present a new framework to deal with it through a visual hierarchical context-based reasoning. Current reasoning methods use the fine-grained labels from images' objects and their interactions to predict labels to new objects. Our framework modifies this current information flow. It goes beyond and is independent of the fine-grained labels from the objects to define the image context. It takes into account the hierarchical interactions between diffe…
