C3-STISR:三线索场景文本图像超分辨率
C3-STISR: Scene Text Image Super-resolution with Triple Clues
摘要:
场景文本图像超分辨率(STISR)是从低分辨率场景文本图像中进行文本识别的重要预处理任务。最新的方法使用识别器的反馈作为线索来指导超分辨率。然而,直接使用识别线索有两个问题:1)兼容性。它是概率分布的形式,与像素级任务STISR有明显的模态差距; 2)不准确性。它通常包含错误的信息,从而会误导主任务并降低超分辨率性能。在本文中,我们提出了一种新的方法C3-STISR,联合利用识别器的反馈,视觉和语言信息作为线索,指导超分辨率。其中,视觉线索来自识别器预测的文本图像,信息量大,与STISR任务兼容性强;而语言线索由预先训练好的字符级语言模型生成,能够对预测文本进行修正。我们设计了有效的三重交叉模态线索提取和融合机制,为超分辨率提供全面统一的指导。在TextZoom上进行的大量实验表明,C3-STISR方法在保真度和识别性能上优于SOTA方法。代码可在https://github.com/zhaominyiz/C3-STISR上找到。
英文摘要:
Scene text image super-resolution (STISR) has been regarded as an important pre-processing task for text recognition from low-resolution scene text images. Most recent approaches use the recognizer's feedback as clues to guide super-resolution. However, directly using recognition clue has two problems: 1) Compatibility. It is in the form of probability distribution, has an obvious modal gap with STISR - a pixel-level task; 2) Inaccuracy. it usually contains wrong information, thus will mislead the main task and degrade super-resolution performance. In this paper, we present a novel method C3-STISR that jointly exploits the recognizer's feedback, visual and linguistical information as clues to guide super-resolution. Here, visual clue is from the images of texts predicted by the recognizer, which is informative and more compatible with the STISR task; while linguistical clue is generated by a pre-trained character-level language model, which is able to correct the predicted texts. We design effective extraction and fusion mechanisms for the triple cross-modal clues to generate a comprehensive and unified guidance for super-resolution. Extensive experiments on TextZoom show that C3-STISR outperforms the SOTA methods in fidelity and recognition performance. Code is available in https://github.com/zhaominyiz/C3-STISR.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。