arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.01422cs.CV
收藏

MDiff4STR:用于场景文本识别的掩码扩散模型

MDiff4STR: Mask Diffusion Model for Scene Text Recognition

Yongkun Du, Miaomiao Zhao, Songlin Fan, Zhineng Chen, Caiyan Jia, Yu-Gang Jiang

AI总结:

MDiff4STR通过改进的掩码扩散模型在场景文本识别中实现更高的准确性和快速推断。

摘要:

掩码扩散模型(MDMs)最近 emerged 作为视觉-语言任务中替代自回归模型(ARMs)的有希望的替代方案,得益于其在效率和准确性之间灵活的平衡。在本文中,我们首次将MDMs引入场景文本识别(STR)任务。我们证明,尽管MDMs在准确性上落后于ARMs,但其在识别效率上有所提高。为弥合这一差距,我们提出了MDiff4STR,一种通过两种关键改进策略增强的掩码扩散模型,这些策略专门针对STR进行了优化。具体来说,我们识别出在将MDMs应用于STR时存在两个关键挑战:训练和推断之间的噪声差距,以及推断过程中过度自信的预测。这两个问题都显著阻碍了MDMs的性能。为了缓解第一个问题,我们开发了六种噪声策略,使训练与推断行为更好地对齐。对于第二个问题,我们提出了一种令牌替换噪声机制,提供了一种非掩码噪声类型,鼓励模型重新考虑并修正过于自信但错误的预测。我们对MDiff4STR在标准和具有挑战性的STR基准上的广泛评估涵盖了多样化的场景,包括不规则、艺术、遮挡和中文文本,以及是否使用预训练。在这些设置中,MDiff4STR始终优于流行的STR模型,在准确性上超越了最先进的ARMs,同时保持快速的推断速度,仅需三个去噪步骤。代码:https://github.com/Topdu/OpenOCR。

英文摘要:

Mask Diffusion Models (MDMs) have recently emerged as a promising alternative to auto-regressive models (ARMs) for vision-language tasks, owing to their flexible balance of efficiency and accuracy. In this paper, for the first time, we introduce MDMs into the Scene Text Recognition (STR) task. We show that vanilla MDM lags behind ARMs in terms of accuracy, although it improves recognition efficiency. To bridge this gap, we propose MDiff4STR, a Mask Diffusion model enhanced with two key improvement strategies tailored for STR. Specifically, we identify two key challenges in applying MDMs to STR: noising gap between training and inference, and overconfident predictions during inference. Both significantly hinder the performance of MDMs. To mitigate the first issue, we develop six noising strategies that better align training with inference behavior. For the second, we propose a token-replacement noise mechanism that provides a non-mask noise type, encouraging the model to reconsider and revise overly confident but incorrect predictions. We conduct extensive evaluations of MDiff4STR on both standard and challenging STR benchmarks, covering diverse scenarios including irregular, artistic, occluded, and Chinese text, as well as whether the use of pretraining. Across these settings, MDiff4STR consistently outperforms popular STR models, surpassing state-of-the-art ARMs in accuracy, while maintaining fast inference with only three denoising steps. Code: https://github.com/Topdu/OpenOCR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑