arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.09661cs.CV
收藏

从二到一:一种新的基于视觉语言建模网络的场景文本识别器

From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network

University of Science and Technology of China(中国科学技术大学);Huawei Cloud & AI(华为云与AI)

Yuxin Wang, Hongtao Xie, Shancheng Fang, Jing Wang, Shenggao Zhu, Yongdong Zhang

摘要:

本文摒弃了占主导地位的复杂语言模型,重新思考场景文本识别中的语言学习过程。不同于以往的方法考虑视觉和语言信息在两个独立的结构,我们提出了一个视觉语言建模网络(VisionLAN),它认为视觉和语言信息作为一个联合体,直接与语言能力的视觉模型。特别地,我们在训练阶段引入了字符遮挡特征图的文本识别。这种操作指导视觉模型不仅使用字符的视觉纹理,而且还使用视觉上下文中的语言信息进行识别时,视觉线索混淆(例如,遮挡,噪声等)。由于语言信息与视觉特征一起获得,而不需要额外的语言模型,VisionLAN将速度显著提高了39%,并自适应地考虑语言信息,以增强视觉特征,从而实现准确识别。此外,一个遮挡场景文本(OST)数据集提出了评估的情况下,失踪的字符明智的视觉线索的性能。几个基准测试的最新结果证明了我们的有效性。代码和数据集可在https://github.com/wangyuxin87/VisionLAN上获得。

英文摘要:

In this paper, we abandon the dominant complex language model and rethink the linguistic learning process in the scene text recognition. Different from previous methods considering the visual and linguistic information in two separate structures, we propose a Visual Language Modeling Network (VisionLAN), which views the visual and linguistic information as a union by directly enduing the vision model with language capability. Specially, we introduce the text recognition of character-wise occluded feature maps in the training stage. Such operation guides the vision model to use not only the visual texture of characters, but also the linguistic information in visual context for recognition when the visual cues are confused (e.g. occlusion, noise, etc.). As the linguistic information is acquired along with visual features without the need of extra language model, VisionLAN significantly improves the speed by 39% and adaptively considers the linguistic information to enhance the visual features for accurate recognition. Furthermore, an Occlusion Scene Text (OST) dataset is proposed to evaluate the performance on the case of missing character-wise visual cues. The state of-the-art results on several benchmarks prove our effectiveness. Code and dataset are available at https://github.com/wangyuxin87/VisionLAN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑