This paper addresses the problem of learning word image representations: given the cropped image of a word, we are interested in finding a descriptive, robust, and compact fixed-length representation. Machine learning techniques can then be supplied with these representations to produce models useful for word retrieval or recognition tasks. Although many works have focused on the machine learning aspect once a global representation has been produced, little work has been devoted to the construction of those base image representations: most works use standard coding and aggregation techniques directly on top of standard computer vision features such as SIFT or HOG.
We propose to learn local mid-level features suitable for building word image representations. These features are learnt by leveraging character bounding box annotations on a small set of training images. However, contrary to other approaches that use character bounding box information, our approach does not rely on detecting the individual characters explicitly at testing time. Our local mid-level features can then be aggregated to produce a global word image signature. When pairing these features with the recent word attributes framework of Almazán et al., we obtain results comparable with or better than the state-of-the-art on matching and recognition tasks using global descriptors of only 96 dimensions.
Learning Mid-level Words on Riemannian Manifold for Action Recognition
Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS)(中国科学院智能信息处理重点实验室);Institute of Computing Technology, CAS(中国科学院计算技术研究所)
MID: A Self-supervised Multimodal Iterative Denoising Framework
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University(自动化与智能感知学院,上海交通大学);Key Laboratory of System Control and Information Processing, Ministry of Education of China(系统控制与信息处理重点实验室,中华人民共和国教育部)
HMM-based Indic Handwritten Word Recognition using Zone Segmentation
Indian Institute of Technology Roorkee(印度理工学院鲁尔基分校);Institute of Engineering & Management, Kolkata(加尔各答工程与管理学院);Indian Statistical Institute, Kolkata(印度统计学院加尔各答分校)
Contrastive Learning for Weakly Supervised Phrase Grounding
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);NVIDIA(英伟达);Bar Ilan University(巴伊兰大学)
短语基础,将图像区域与字幕词相关联的问题,是视觉语言任务的重要组成部分。我们表明,短语接地可以通过优化字区域的注意,以最大限度地提高图像和字幕词之间的互信息的下限。给定成对的图像和字幕,我们最大限度地提高了注意力加权区域和相应字幕中的单词的兼容性,相比于非对应的图像和字幕对。其核心思想是通过语言模型指导的词替换来构造有效的负面字幕。与来自训练数据的随机采样负数相比,使用我们的负数进行训练会在准确性方面产生$\sim10\%$绝对增益。我们在COCO-Captions上训练的弱监督短语接地模型显示了$5.7\%$的健康增益,以在Flickr 30 K Entities基准上实现$76.7\%$准确性。
Weakly Supervised Segmentation and Classification of Alpha-Synuclein Aggregates in Brightfield Midbrain Images
Computational Biology, Ecole Normale Sup\'erieure, PSL University, Paris, France. 2 AP-HP, Hôpital Henri Mondor-Albert Chenevier, Service de Neurologie, F-94010 Créteil, France. 3 INSERM U955, Institut Mondor de Recherche Biomédicale, UPEC, Equipe NeuroPsychologie Interventionnelle, F-94010 Creteil, France. 4 Département d'Etudes Cognitives, École normale supérieure, PSL University, 75005 Paris, France. 5 NeurATRIS, Créteil, France. 6 Nuffield Department of Clinical Neurosciences;the Queen’s College, University of Oxford, UK. 7 Department of Women’s;Children’s Health, Karolinska Institutet, Sweden. 8 Churchill College, Cambridge University, CB30DS UK.