DianJin-OCR-R1: 通过推理与工具交替的视觉语言模型增强OCR能力
DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Model
AI总结：
DianJin-OCR-R1通过推理与工具交替的视觉语言模型框架，提升OCR识别的准确性和上下文理解能力。
摘要：
最近视觉语言模型（VLMs）的进展使端到端文档解析和理解成为可能，在多种光学字符识别（OCR）任务上实现了强劲性能。然而，VLMs由于过度依赖语言先验，容易生成输入图像中不存在的词语。相比之下，传统OCR模型因其架构专门针对特定识别任务，通常能实现更强的细粒度视觉感知，但通常缺乏在更具挑战性情况下所需的上下文语义理解和推理能力。为弥合这一差距，我们提出了DianJin-OCR-R1，一种增强推理的识别框架，通过推理与工具交替的范式训练VLMs。我们的DianJin-OCR-R1模型首先通过其自身的OCR能力识别输入图像中的内容，然后调用其他专家模型以获得额外结果作为参考。之后，它被引导重新查看图像，并将自身识别的内容与其他结果进行比较，以发现错误或遗漏。最后，它整合所有可用证据生成更准确的输出。这种设计使模型能够学习如何隐式地重新聚焦于视觉输入，并有效利用其他专家模型的结果以提高性能。我们在ReST和OmniDocBench上评估了我们的DianJin-OCR-R1模型，结果在两个数据集上均优于其非推理对照模型和专家模型，证明了我们方法的有效性。
英文摘要：
Recent advances in vision-language models (VLMs) have enabled end-to-end document parsing and understanding, achieving strong performance on diverse optical character recognition (OCR) tasks. However, VLMs are prone to generate words that do not exist in the input image due to over-reliance on language priors. By contrast, traditional OCR models, whose architectures are tailored for specific recognition tasks, often achieve stronger fine-grained visual perception with fewer hallucinations, but they typically lack the contextual semantic understanding and reasoning capabilities needed in more challenging cases. To bridge this gap, we propose DianJin-OCR-R1, a reasoning-enhanced framework for recognition that trains VLMs in a reasoning-and-tool interleaved paradigm. Our DianJin-OCR-R1 model first recognizes the content in the input image through its own OCR capabilities, and then calls other expert models for extra results as references. After that, it is guided to "look again" at the image and compare its own recognized content with other results to find errors or omissions. Finally, it integrates all available evidence to generate a more accurate output. This design empowers the model to learn how to implicitly re-focus on the visual input and effectively leverage the results of other expert models for better performance. We evaluate our DianJin-OCR-R1 model on ReST and OmniDocBench, where it consistently outperforms both its non-reasoning counterparts and expert models, demonstrating the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
