LEWIS:词图像及其语义的潜在嵌入
LEWIS: Latent Embeddings for Word Images and their Semantics
摘要:
这项工作的目标是将语义纳入自然图像中的文本识别和检索任务。虽然文本识别和检索近年来受到了很多关注,但以前的工作都集中在识别或检索用作查询的完全相同的词,而不考虑语义。在本文中,我们提出了以下问题:\emph{can we predict semantic concepts directly from a word image, without explicitly trying to transcribe the word image or its characters at any point?}为了这个目标,我们提出了一个卷积神经网络(CNN)与加权排名损失的目标,确保相关的查询图像的概念排名领先于那些不相关的。这也可以解释为学习一个欧几里德空间,其中单词图像和概念联合嵌入。该模型是以端到端的方式学习的,从图像像素到语义概念,使用从词汇数据库(WordNet)中挖掘的综合生成的单词图像和概念的数据集。我们的研究结果表明,尽管任务的复杂性,单词图像和概念确实可以与高准确度的关联
英文摘要:
The goal of this work is to bring semantics into the tasks of text recognition and retrieval in natural images. Although text recognition and retrieval have received a lot of attention in recent years, previous works have focused on recognizing or retrieving exactly the same word used as a query, without taking the semantics into consideration. In this paper, we ask the following question: \emph{can we predict semantic concepts directly from a word image, without explicitly trying to transcribe the word image or its characters at any point?} For this goal we propose a convolutional neural network (CNN) with a weighted ranking loss objective that ensures that the concepts relevant to the query image are ranked ahead of those that are not relevant. This can also be interpreted as learning a Euclidean space where word images and concepts are jointly embedded. This model is learned in an end-to-end manner, from image pixels to semantic concepts, using a dataset of synthetically generated word images and concepts mined from a lexical database (WordNet). Our results show that, despite the complexity of the task, word images and concepts can indeed be associated with a high degree of accuracy
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。