arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.21053cs.CV
收藏

OCR-Agent: 具有能力和记忆反思的代理OCR

OCR-Agent: Agentic OCR with Capability and Memory Reflection

Southwest Minzu University(西南民族大学);AI Geeks

Shimin Wen, Zeyu Zhang, Xingdou Bian, Hongjie Zhu, Lulu He, Layi Shama, Daji Ergu, Ying Cai

AI总结:

OCR-Agent通过能力反思和记忆反思机制,提升VLMs的推理鲁棒性,实现更稳定的答案质量改进。

摘要:

大型视觉-语言模型(VLMs)通过迭代优化方法在复杂的视觉理解任务中展示了显著的潜力。然而,这些模型通常缺乏有效的自我纠正机制,使它们难以独立纠正认知偏差。因此,在多轮修订中,它们往往陷入重复和无效的尝试,无法实现答案质量的稳定提升。为了解决这个问题,我们提出了一种新的迭代自我纠正框架,赋予模型两种关键能力:能力反思和记忆反思。该框架引导模型首先通过能力反思诊断错误并生成纠正计划,然后利用记忆反思回顾过去的尝试以避免重复并探索新解决方案,最后通过严格的再推理优化答案。在具有挑战性的OCRBench v2基准测试中,OCR-Agent在英语子集上比当前开源SOTA模型InternVL3-8B高出+2.0,在中文子集上高出+1.2,同时在视觉理解(79.9)和推理(66.5)方面取得最先进的结果,超越了甚至更大的微调模型。我们的方法表明,结构化的自我意识反思可以显著提高VLMs的推理鲁棒性,而无需额外训练。代码:https://github.com/AIGeeksGroup/OCR-Agent

英文摘要:

Large Vision-Language Models (VLMs) have demonstrated significant potential on complex visual understanding tasks through iterative optimization methods.However, these models generally lack effective self-correction mechanisms, making it difficult for them to independently rectify cognitive biases. Consequently, during multi-turn revisions, they often fall into repetitive and ineffective attempts, failing to achieve stable improvements in answer quality.To address this issue, we propose a novel iterative self-correction framework that endows models with two key capabilities: Capability Reflection and Memory Reflection. This framework guides the model to first diagnose errors and generate a correction plan via Capability Reflection, then leverage Memory Reflection to review past attempts to avoid repetition and explore new solutions, and finally, optimize the answer through rigorous re-reasoning. Experiments on the challenging OCRBench v2 benchmark show that OCR-Agent outperforms the current open-source SOTA model InternVL3-8B by +2.0 on English and +1.2 on Chinese subsets, while achieving state-of-the-art results in Visual Understanding (79.9) and Reasoning (66.5) - surpassing even larger fine-tuned models. Our method demonstrates that structured, self-aware reflection can significantly enhance VLMs' reasoning robustness without additional training. Code: https://github.com/AIGeeksGroup/OCR-Agent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑