Few-shot learning aims to generalize to novel classes with only a few samples with class labels. Research in few-shot learning has borrowed techniques from transfer learning, metric learning, meta-learning, and Bayesian methods. These methods also aim to train models from limited training samples, and while encouraging performance has been achieved, they often fail to generalize to novel domains. Many of the existing meta-learning methods rely on training data for which the base classes are sampled from the same domain as the novel classes used for meta-testing. However, in many applications in the industry, such as document classification, collecting large samples of data for meta-learning is infeasible or impossible. While research in the field of the cross-domain few-shot learning exists, it is mostly limited to computer vision. To our knowledge, no work yet exists that examines the use of few-shot learning for classification of semi-structured documents (scans of paper documents) generated as part of a business workflow (forms, letters, bills, etc.). Here the domain shift is significant, going from natural images to the semi-structured documents of interest. In this work, we address the problem of few-shot document image classification under domain shift. We evaluate our work by extensive comparisons with existing methods. Experimental results demonstrate that the proposed method shows consistent improvements on the few-shot classification performance under domain shift.
Class-Agnostic Region-of-Interest Matching in Document Images
VCIP \& TMCC \& DISSec, College of Computer Science \& College of Cryptology；Cyber Science, Nankai University, China , Institute of Information Engineering, Chinese Academy of Science, China School of Cyber Security, University of Chinese Academy of Science, China Department of Computer Science；Technology, Tsinghua University, China
Cinnamon AI（肉桂人工智能公司）；Hung Yen University of Technology and Education（兴安技术教育大学）；Deakin University（迪肯大学）
Successful Artificial Intelligence systems often require numerous labeled data to extract information from document images. In this paper, we investigate the problem of improving the performance of Artificial Intelligence systems in understanding document images, especially in cases where training data is limited. We address the problem by proposing a novel finetuning method using reinforcement learning. Our approach treats the Information Extraction model as a policy network and uses policy gradient training to update the model to maximize combined reward functions that complement the traditi…
GDI-Bench: A Benchmark for General Document Intelligence with Vision and Reasoning Decoupling
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Zhejiang University（浙江大学）；School of Science and Engineering, The Chinese University of Hong Kong（香港中文大学科学与工程学院）；Fudan University（复旦大学）；Shanghai Innovation Institute（上海创新研究院）
FCMBench-Video: Benchmarking Document Video Intelligence
AI Lab, Qifu Technology（启赋科技AI实验室）；College of Future Information Technology, Fudan University（复旦大学未来信息学院）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；Pazhou Lab（琶洲实验室）
Pretraining has proven successful in Document Intelligence tasks where deluge of documents are used to pretrain the models only later to be finetuned on downstream tasks. One of the problems of the pretraining approaches is the inconsistent usage of pretraining data with different OCR engines leading to incomparable results between models. In other words, it is not obvious whether the performance gain is coming from diverse usage of amount of data and distinct OCR engines or from the proposed models. To remedy the problem, we make public the OCR annotations for IDL documents using commercial O…
