A Multi-oriented Chinese Keyword Spotter Guided by Text Line Detection

Robust End-to-End Offline Chinese Handwriting Text Page Spotter with Text Kernel

Offline Chinese handwriting text recognition is a long-standing research topic in the field of pattern recognition. In previous studies, text detection and recognition are separated, which leads to the fact that text recognition is highly dependent on the detection results. In this paper, we propose a robust end-to-end Chinese text page spotter framework. It unifies text detection and text recognition with text kernel that integrates global text feature information to optimize the recognition from multiple scales, which reduces the dependence of detection and improves the robustness of the sys…

Generating Chinese Poetry from Images via Concrete and Abstract Information

In recent years, the automatic generation of classical Chinese poetry has made great progress. Besides focusing on improving the quality of the generated poetry, there is a new topic about generating poetry from an image. However, the existing methods for this topic still have the problem of topic drift and semantic inconsistency, and the image-poem pairs dataset is hard to be built when training these models. In this paper, we extract and integrate the Concrete and Abstract information from images to address those issues. We proposed an infilling-based Chinese poetry generation model which ca…

Chinese Street View Text: Large-scale Chinese Text Reading with Partially Supervised Learning

Most existing text reading benchmarks make it difficult to evaluate the performance of more advanced deep learning models in large vocabularies due to the limited amount of training data. To address this issue, we introduce a new large-scale text reading benchmark dataset named Chinese Street View Text (C-SVT) with 430,000 street view images, which is at least 14 times as large as the existing Chinese text reading benchmarks. To recognize Chinese text in the wild while keeping large-scale datasets labeling cost-effective, we propose to annotate one part of the CSVT dataset (30,000 images) in l…

GoMatching：一种基于长短期匹配的视频文本定位简单基线

GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matching

针对现有视频文本定位器识别能力有限的问题，研究提出GoMatching基线，改造图像文本定位器并加入重评分头和长短期匹配模块，在多个数据集上刷新性能且节省训练成本。

TextFormer：一种基于查询的混合监督端到端文本识别器

TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision

TextFormer提出基于查询的Transformer端到端文本识别器，通过联合建模分类、分割与识别，并利用自适应全局聚合模块和混合监督，在多个中英文基准上取得领先性能。

Visually grounded learning of keyword prediction from untranscribed speech

During language acquisition, infants have the benefit of visual cues to ground spoken language. Robots similarly have access to audio and visual sensors. Recent work has shown that images and spoken captions can be mapped into a meaningful common space, allowing images to be retrieved using speech and vice versa. In this setting of images paired with untranscribed spoken captions, we consider whether computer vision systems can be used to obtain textual labels for the speech. Concretely, we use an image-to-words multi-label visual classifier to tag images with soft textual labels, and then tra…