In this work, for the first time, we address the problem of universal cross-domain retrieval, where the test data can belong to classes or domains which are unseen during training. Due to dynamically increasing number of categories and practical constraint of training on every possible domain, which requires large amounts of data, generalizing to both unseen classes and domains is important. Towards that goal, we propose SnMpNet (Semantic Neighbourhood and Mixture Prediction Network), which incorporates two novel losses to account for the unseen classes and domains encountered during testing. Specifically, we introduce a novel Semantic Neighborhood loss to bridge the knowledge gap between seen and unseen classes and ensure that the latent space embedding of the unseen classes is semantically meaningful with respect to its neighboring classes. We also introduce a mix-up based supervision at image-level as well as semantic-level of the data for training with the Mixture Prediction loss, which helps in efficient retrieval when the query belongs to an unseen domain. These losses are incorporated on the SE-ResNet50 backbone to obtain SnMpNet. Extensive experiments on two large-scale datasets, Sketchy Extended and DomainNet, and thorough comparisons with state-of-the-art justify the effectiveness of the proposed model.
V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
Tsinghua University(清华大学);University of Central Florida(中央佛罗里达大学);Fudan University(复旦大学);The Australian National University(澳大利亚国立大学);The University of Hong Kong(香港大学);Pengcheng Laboratory(鹏城实验室);Central South University(中南大学)
Generalizing AI-driven Assessment of Immunohistochemistry across Immunostains and Cancer Types: A Universal Immunohistochemistry Analyzer
Lunit;Samsung Medical Center(三星医学中心);Sungkyunkwan University School of Medicine(成均馆大学医学院);Seoul National University Bundang Hospital(首尔大学盆唐医院);Ajou University School of Medicine(亚洲大学医学院);Ajou University Graduate School of Medicine(亚洲大学医学研究生院)
Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion
Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval
School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);Zhongguancun Academy(中关村实验室);Xiaohongshu(小红书);Southeast Academy of Information Technology, Beijing Institute of Technology(北京理工大学东南信息科学研究院);Zhongguancun Laboratory(中关村实验室)
School of Traffic and Transportation, Beijing Jiaotong University(交通与运输学院,北京交通大学);Foundation Modal Research Center, Institute of Automation, Chinese Academy of Sciences(基础模态研究中心,自动化研究所);Queen Mary School Hainan, Beijing University of Posts and Telecommunications(海南女王学院,北京邮电大学);Department of Computer Science, NUS School of Computing(计算机科学系,国立大学计算机学院)
VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval
Beijing University of Posts and Telecommunications(北京邮电大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);The Hong Kong Polytechnic University(香港理工大学)
MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval
Beijing University of Posts and Telecommunications(北京邮电大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);The Hong Kong Polytechnic University(香港理工大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiaotong University(上海交通大学)
UniIR: Training and Benchmarking Universal Multimodal Information Retrievers
University of Waterloo(滑铁卢大学);Georgia Institute of Technology(佐治亚理工学院);Hong Kong University of Science and Technology(香港科技大学);Google DeepMind(谷歌DeepMind)
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval
Zhejiang University(浙江大学);Carnegie Mellon University(卡内基梅隆大学);University of Washington(华盛顿大学);Dalian University of Technology(大连理工大学);DAMO Academy, Alibaba Group(阿里巴巴达摩院);Microsoft Research(微软研究院)