Academic documents are packed with texts, equations, tables, and figures, requiring comprehensive understanding for accurate Optical Character Recognition (OCR). While end-to-end OCR methods offer improved accuracy over layout-based approaches, they often grapple with significant repetition issues, especially with complex layouts in Out-Of-Domain (OOD) documents.To tackle this issue, we propose LOCR, a model that integrates location guiding into the transformer architecture during autoregression. We train the model on a dataset comprising over 77M text-location pairs from 125K academic document pages, including bounding boxes for words, tables and mathematical symbols. LOCR adeptly handles various formatting elements and generates content in Markdown language. It outperforms all existing methods in our test set constructed from arXiv, as measured by edit distance, BLEU, METEOR and F-measure.LOCR also reduces repetition frequency from 4.4% of pages to 0.5% in the arXiv dataset, from 13.2% to 1.3% in OOD quantum physics documents and from 8.1% to 1.8% in OOD marketing documents. Additionally, LOCR features an interactive OCR mode, facilitating the generation of complex documents through a few location prompts from human.
DDVR-Net: Degradation-Aware Decoupling and Visibility-Guided Restoration for Historical Document Recognition
Institute of National Security, Minzu University of China(中央民族大学国家安全研究院);School of Information Engineering, Minzu University of China(中央民族大学信息工程学院);School of Artificial Intelligence and Computer Science, North China University of Technology(北方工业大学人工智能与计算机学院)
MultiAnimate: Pose-Guided Image Animation Made Extensible
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(人工智能安全国家重点实验室,计算技术研究所,中国科学院);ShanghaiTech University(上海科技大学);Shanghai Jiao Tong University(上海交通大学)
GLYPH-SR: Can We Achieve Both High-Quality Image Super-Resolution and High-Fidelity Text Recovery via VLM-guided Latent Diffusion Model?
Department of Artificial Intelligence(人工智能系);Kyungpook National University(庆尚国立大学);Department of Electrical and Computer Engineering(电气电子工程系);Queen’s University(皇后大学);Department of Information and Communications Engineering(信息与通信工程系);Pukyong National University(浦项国立大学)
Minecraft-ify: Minecraft Style Image Generation with Text-guided Image Editing for In-Game Application
School of Art and Technology, Chung-Ang University(中央大学艺术与技术学院);VIVE STUDIOS(VIVE工作室);GSAIM, Chung-Ang University(中央大学GSAIM);Department of Computer Science and Engineering, Chung-Ang University(中央大学计算机科学与工程系)