TextFormer：一种基于查询的混合监督端到端文本识别器
TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision
AI总结：
TextFormer提出基于查询的Transformer端到端文本识别器，通过联合建模分类、分割与识别，并利用自适应全局聚合模块和混合监督，在多个中英文基准上取得领先性能。
摘要：
端到端文本识别是一项重要的计算机视觉任务，旨在将场景文本检测与识别整合到一个统一框架中。典型方法严重依赖感兴趣区域（RoI）操作来提取局部特征，并依赖复杂的后处理步骤来生成最终预测。为解决这些局限，我们提出TextFormer，一种基于查询的Transformer架构端到端文本识别器。具体而言，TextFormer利用每个文本实例的查询嵌入，建立在图像编码器和文本解码器之上，学习多任务建模的联合语义理解。它允许分类、分割和识别分支相互训练与优化，从而在不牺牲灵活性或简洁性的前提下实现更深层的特征共享。此外，我们设计了一个自适应全局聚合（AGG）模块，将全局特征转换为序列特征以读取任意形状文本，克服了RoI操作的次优化问题。进一步地，通过混合监督，从弱标注到完整标签中利用潜在的语料信息，进一步提升了文本检测和端到端文本识别结果。在多个双语（即英文和中文）基准上的大量实验证明了我们方法的优越性。尤其在TDA-ReCTS数据集上，TextFormer在1-NED指标上超越最先进方法13.2%。
英文摘要：
End-to-end text spotting is a vital computer vision task that aims to integrate scene text detection and recognition into a unified framework. Typical methods heavily rely on Region-of-Interest (RoI) operations to extract local features and complex post-processing steps to produce final predictions. To address these limitations, we propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. Specifically, using query embedding per text instance, TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing without sacrificing flexibility or simplicity. Additionally, we design an Adaptive Global aGgregation (AGG) module to transfer global features into sequential features for reading arbitrarily-shaped texts, which overcomes the sub-optimization problem of RoI operations. Furthermore, potential corpus information is utilized from weak annotations to full labels through mixed supervision, further improving text detection and end-to-end text spotting results. Extensive experiments on various bilingual (i.e., English and Chinese) benchmarks demonstrate the superiority of our method. Especially on TDA-ReCTS dataset, TextFormer surpasses the state-of-the-art method in terms of 1-NED by 13.2%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
