In this paper, we present the Hierarchy-of-Visual-Words (HoVW), a novel trademark image retrieval (TIR) method that decomposes images into simpler geometric shapes and defines a descriptor for binary trademark image representation by encoding the hierarchical arrangement of component shapes. The proposed hierarchical organization of visual data stores each component shape as a visual word. It is capable of representing the geometry of individual elements and the topology of the trademark image, making the descriptor robust against linear as well as to some level of nonlinear transformation. Experiments show that HoVW outperforms previous TIR methods on the MPEG-7 CE-1 and MPEG-7 CE-2 image databases.
Madhav Institute of Technology and Science（马德hav理工学院）
Trademark Image Retrieval is playing a vital role as a part of CBIR System. Trademark is of great significance because it carries the status value of any company. To retrieve such a fake or copied trademark we design a retrieval system which is based on hybrid techniques. It contains a mixture of two different feature vector which combined together to give a suitable retrieval system. In the proposed system we extract the corner feature which is applied on an edge pixel image. This feature is used to extract the relevant image and to more purify the result we apply other feature which is the i…
Large-scale trademark retrieval is an important content-based image retrieval task. A recent study shows that off-the-shelf deep features aggregated with Regional-Maximum Activation of Convolutions (R-MAC) achieve state-of-the-art results. However, R-MAC suffers in the presence of background clutter/trivial regions and scale variance, and discards important spatial information. We introduce three simple but effective modifications to R-MAC to overcome these drawbacks. First, we propose the use of both sum and max pooling to minimise the loss of spatial information. We also employ domain-specif…
Queensland University of Technology（昆士兰科技大学）；TrademarkVision（商标视觉）
The demand for large-scale trademark retrieval (TR) systems has significantly increased to combat the rise in international trademark infringement. Unfortunately, the ranking accuracy of current approaches using either hand-crafted or pre-trained deep convolution neural network (DCNN) features is inadequate for large-scale deployments. We show in this paper that the ranking accuracy of TR systems can be significantly improved by incorporating hard and soft attention mechanisms, which direct attention to critical information such as figurative elements and reduce attention given to distracting …
Off-the-shelf convolutional neural network features achieve outstanding results in many image retrieval tasks. However, their invariance to target data is pre-defined by the network architecture and training data. Existing image retrieval approaches require fine-tuning or modification of pre-trained networks to adapt to variations unique to the target data. In contrast, our method enhances the invariance of off-the-shelf features by aggregating features extracted from images augmented at test-time, with augmentations guided by a policy learned through reinforcement learning. The learned policy…
Logo retrieval is a challenging problem since the definition of similarity is more subjective compared to image retrieval tasks and the set of known similarities is very scarce. To tackle this challenge, in this paper, we propose a simple but effective segment-based augmentation strategy to introduce artificially similar logos for training deep networks for logo retrieval. In this novel augmentation strategy, we first find segments in a logo and apply transformations such as rotation, scaling, and color change, on the segments, unlike the conventional image-level augmentation strategies. Moreo…
Trademark retrieval (TR) has become an important yet challenging problem due to an ever increasing trend in trademark applications and infringement incidents. There have been many promising attempts for the TR problem, which, however, fell impracticable since they were evaluated with limited and mostly trivial datasets. In this paper, we provide a large-scale dataset with benchmark queries with which different TR approaches can be evaluated systematically. Moreover, we provide a baseline on this benchmark using the widely-used methods applied to TR in the literature. Furthermore, we identify a…
Deep neural networks trained for classification have been found to learn powerful image representations, which are also often used for other tasks such as comparing images w.r.t. their visual similarity. However, visual similarity does not imply semantic similarity. In order to learn semantically discriminative features, we propose to map images onto class embeddings whose pair-wise dot products correspond to a measure of semantic similarity between classes. Such an embedding does not only improve image retrieval results, but could also facilitate integrating semantics for other tasks, e.g., n…
School of Information and Communication Engineering（信息与通信工程学院）；University of Electronic Science and Technology of China（电子科技大学）
Recently, deep supervised hashing methods have become popular for large-scale image retrieval task. To preserve the semantic similarity notion between examples, they typically utilize the pairwise supervision or the triplet supervised information for hash learning. However, these methods usually ignore the semantic class information which can help the improvement of the semantic discriminative ability of hash codes. In this paper, we propose a novel hierarchy neighborhood discriminative hashing method. Specifically, we construct a bipartite graph to build coarse semantic neighbourhood relation…
City University of Hong Kong（香港城市大学）；Tencent AI Lab（腾讯人工智能实验室）；Shenzhen University（深圳大学）
Deep hashing models have been proposed as an efficient method for large-scale similarity search. However, most existing deep hashing methods only utilize fine-level labels for training while ignoring the natural semantic hierarchy structure. This paper presents an effective method that preserves the classwise similarity of full-level semantic hierarchy for large-scale image retrieval. Experiments on two benchmark datasets show that our method helps improve the fine-level retrieval performance. Moreover, with the help of the semantic hierarchy, it can produce significantly better binary codes f…
The Visual Target Matters: Learning across the Visual Hierarchy for Brain-to-Image Retrieval
Chongqing University of Posts and Telecommunications（重庆邮电大学）；Shanghai Jiaotong University（上海交通大学）；Beijing University of Chemical Technology（北京化工大学）；The Hong Kong Polytechnic University（香港理工大学）
MIRAGE: Runtime Scheduling for Multi-Vector Image Retrieval with Hierarchical Decomposition
School of Computer Science, Peking University（北京大学计算机科学学院）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）；School of Information, Renmin University of China（中国人民大学信息学院）；School of Integrated Circuit Science and Engineering, Beihang University（北京航空航天大学集成电路科学与工程学院）
The recent advances brought by deep learning allowed to improve the performance in image retrieval tasks. Through the many convolutional layers, available in a Convolutional Neural Network (CNN), it is possible to obtain a hierarchy of features from the evaluated image. At every step, the patches extracted are smaller than the previous levels and more representative. Following this idea, this paper introduces a new detector applied on the feature maps extracted from pre-trained CNN. Specifically, this approach lets to increase the number of features in order to increase the performance of the …
Conservatoire National des Arts et Métiers（法国国立工艺学院）；Coexya（Coexya公司）；Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；ISIR（智能系统与机器人研究所）
Image Retrieval is commonly evaluated with Average Precision (AP) or Recall@k. Yet, those metrics, are limited to binary labels and do not take into account errors' severity. This paper introduces a new hierarchical AP training method for pertinent image retrieval (HAP-PIER). HAPPIER is based on a new H-AP metric, which leverages a concept hierarchy to refine AP by integrating errors' importance and better evaluate rankings. To train deep models with H-AP, we carefully study the problem's structure and design a smooth lower bound surrogate combined with a clustering loss that ensures consisten…
Learning to hash is an efficient paradigm for exact and approximate nearest neighbor search from massive databases. Binary hash codes are typically extracted from an image by rounding output features from a CNN, which is trained on a supervised binary similar/ dissimilar task. Drawbacks of this approach are: (i) resulting codes do not necessarily capture semantic similarity of the input data (ii) rounding results in information loss, manifesting as decreased retrieval performance and (iii) Using only class-wise similarity as a target can lead to trivial solutions, simply encoding classifier ou…
Proxy-based metric learning losses are superior to pair-based losses due to their fast convergence and low training complexity. However, existing proxy-based losses focus on learning class-discriminative features while overlooking the commonalities shared across classes which are potentially useful in describing and matching samples. Moreover, they ignore the implicit hierarchy of categories in real-world datasets, where similar subordinate classes can be grouped together. In this paper, we present a framework that leverages this implicit hierarchy by imposing a hierarchical structure on the p…
This paper addresses the problem of very large-scale image retrieval, focusing on improving its accuracy and robustness. We target enhanced robustness of search to factors such as variations in illumination, object appearance and scale, partial occlusions, and cluttered backgrounds - particularly important when search is performed across very large datasets with significant variability. We propose a novel CNN-based global descriptor, called REMAP, which learns and aggregates a hierarchy of deep features from multiple CNN layers, and is trained end-to-end with a triplet loss. REMAP explicitly l…
Class Agnostic Instance-level Descriptor for Visual Instance Search
Computer Science Department, Xiamen University（厦门大学计算机科学系）；Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究所）；School of Computing and Information Systems, Singapore Management University（新加坡管理学院计算机与信息系统学院）
Advanced Technology and Development Center, IIT Kharagpur（印度理工学院卡拉格普尔分校先进技术与开发中心）；IIT Kharagpur（印度理工学院卡拉格普尔分校）
Searching for similar logos in the registered logo database is a very important and tedious task at the trademark office. Speed and accuracy are two aspects that one must attend to while developing a system for retrieval of logos. In this paper, we propose a rough-set based method to quantify the structural information in a logo image that can be used to efficiently index an image. A logo is split into a number of polygons, and for each polygon, we compute the tight upper and lower approximations based on the principles of a rough set. This representation is used for forming feature vectors fo…
Beijing Institute of Technology（北京理工大学）；National University of Singapore（新加坡国立大学）；South China University of Technology（华南理工大学）
Recently, hashing methods have been widely used in large-scale image retrieval. However, most existing hashing methods did not consider the hierarchical relation of labels, which means that they ignored the rich information stored in the hierarchy. Moreover, most of previous works treat each bit in a hash code equally, which does not meet the scenario of hierarchical labeled data. In this paper, we propose a novel deep hashing method, called supervised hierarchical deep hashing (SHDH), to perform hash code learning for hierarchical labeled data. Specifically, we define a novel similarity formu…
Harbin Institute of Technology（哈尔滨工业大学）；JD AI Research（京东AI研究院）
Merchandise categories inherently form a semantic hierarchy with different levels of concept abstraction, especially for fine-grained categories. This hierarchy encodes rich correlations among various categories across different levels, which can effectively regularize the semantic space and thus make predictions less ambiguous. However, previous studies of fine-grained image retrieval primarily focus on semantic similarities or visual similarities. In a real application, merely using visual similarity may not satisfy the need of consumers to search merchandise with real-life images, e.g., giv…
Northeastern University（东北大学）；University of California at Santa Cruz（加州大学圣克鲁兹分校）
Searching for relative mobile user interface (UI) design examples can aid interface designers in gaining inspiration and comparing design alternatives. However, finding such design examples is challenging, especially as current search systems rely on only text-based queries and do not consider the UI structure and content into account. This paper introduces VINS, a visual search framework, that takes as input a UI image (wireframe, high-fidelity) and retrieves visually similar design examples. We first survey interface designers to better understand their example finding process. We then devel…
Hypernymy, textual entailment, and image captioning can be seen as special cases of a single visual-semantic hierarchy over words, sentences, and images. In this paper we advocate for explicitly modeling the partial order structure of this hierarchy. Towards this goal, we introduce a general method for learning ordered representations, and show how it can be applied to a variety of tasks involving images and language. We show that the resulting representations improve performance over current approaches for hypernym prediction and image-caption retrieval.
HASSL: Hierarchy-Aware Self-Supervised Learning Framework for Single Cell Microscopy
TUM.ai, Munich, Germany（慕尼黑工业大学人工智能中心，德国慕尼黑）；Ludwig-Maximilian-University, Munich, Germany（慕尼黑大学，德国慕尼黑）；Technical University of Munich, Munich, Germany（慕尼黑工业大学，德国慕尼黑）；Institute of Stem Cell Research, Helmholtz Munich, German Research Center for Environmental Health, Neuherberg, Germany（干细胞研究所，亥姆霍兹慕尼黑中心，德国环境健康研究中心，德国新赫伯格）；Computational Health Center, Helmholtz Munich - German Research Center for Environmental Health, Neuherberg, Germany（计算健康中心，亥姆霍兹慕尼黑中心 - 德国环境健康研究中心，德国新赫伯格）；Department of Medicine III, Ludwig-Maximilian-University Hospital, Munich, Germany（医学部III，慕尼黑大学医院，德国慕尼黑）；Department of Physics, Ludwig-Maximilian-University, Munich, Germany（物理系，慕尼黑大学，德国慕尼黑）；German Cancer Consortium (DKTK), partner site Munich, Germany（德国癌症联盟（DKTK），慕尼黑合作站点，德国慕尼黑）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML），德国慕尼黑）
SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval
Multimodal Art Projection Research Community（多模态艺术投影研究共同体）；University of Manchester（曼彻斯特大学）；University of Waterloo（滑铁卢大学）；Hong Kong University of Science and Technology（香港科技大学）；Durham University（杜伦大学）
