In this paper, we describe our end-to-end content-based image retrieval system built upon Elasticsearch, a well-known and popular textual search engine. As far as we know, this is the first time such a system has been implemented in eCommerce, and our efforts have turned out to be highly worthwhile. We end up with a novel and exciting visual search solution that is extremely easy to be deployed, distributed, scaled and monitored in a cost-friendly manner. Moreover, our platform is intrinsically flexible in supporting multimodal searches, where visual and textual information can be jointly leveraged in retrieval.
The core idea is to encode image feature vectors into a collection of string tokens in a way such that closer vectors will share more string tokens in common. By doing that, we can utilize Elasticsearch to efficiently retrieve similar images based on similarities within encoded sting tokens. As part of the development, we propose a novel vector to string encoding method, which is shown to substantially outperform the previous ones in terms of both precision and latency.
First-hand experiences in implementing this Elasticsearch-based platform are extensively addressed, which should be valuable to practitioners also interested in building visual search engine on top of Elasticsearch.
Multimodal Neural Machine Translation with Search Engine Based Image Retrieval
College of Application and Technology（应用技术学院）；Shenzhen University（深圳大学）；College of Big Data and Internet（大数据与互联网学院）；Shenzhen Technology University（深圳技术大学）
Imagine Before You Draw: Visual Prompt Engineering for Image Generation
Nanyang Technological University（南洋理工大学）；National University of Singapore（国立新加坡大学）；Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）
VisualPrompter: Semantic-Aware Prompt Optimization with Visual Feedback for Text-to-Image Synthesis
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）
SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机科学学院多媒体信息处理国家重点实验室）；University of Pennsylvania（宾夕法尼亚大学）；University of Electronic Science and Technology of China（电子科技大学）；Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；De Artificial Intelligence Lab（德人工智能实验室）
Visual Prompt Engineering for Vision Language Models in Radiology
Division of Medical Image Computing, German Cancer Research Center, Heidelberg, Germany（德国癌症研究中心医学图像计算部）；Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany（海德堡大学数学与计算机科学学院）；Medical Faculty Heidelberg, University of Heidelberg, Heidelberg, Germany（海德堡大学医学学院）
HyPER-GAN: Hybrid Patch-Based Image-to-Image Translation for Real-Time Photorealism Enhancement in Game Engines
organization= School of Informatics , addressline= Aristotle University of Thessaloniki (AUTH) , city= Thessaloniki , postcode= 54124 , country= Greece
mViSE: A Visual Search Engine for Analyzing Multiplex IHC Brain Tissue Images
Cullen College of Engineering, University of Houston（休斯顿大学塞勒恩工程学院）；The University of Texas McGovern Medical School（德克萨斯大学麦格芬医学学院）；National Institute of Neurological Disorders and Stroke（国家神经疾病与中风研究所）
