arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.08356cs.CV
收藏

通用跨域检索:跨类和跨域的泛化

Universal Cross-Domain Retrieval: Generalizing Across Classes and Domains

Indian Institute of Technology, Kharagpur(印度理工学院卡拉格普尔分校);Indian Institute of Science, Bangalore(印度科学理工学院班加罗尔分校)

Soumava Paul, Titir Dutta, Soma Biswas

摘要:

在这项工作中,我们第一次解决了通用跨域检索的问题,其中测试数据可以属于训练过程中看不到的类或域。由于类别数量的动态增加以及对每个可能领域的训练的实际约束,这需要大量的数据,因此推广到看不见的类别和领域是很重要的。为了实现这一目标,我们提出了SnMpNet(语义邻域和混合预测网络),它包含了两个新的损失,以解释测试过程中遇到的看不见的类和域。具体来说,我们引入了一种新的语义邻域损失来弥合可见和不可见类之间的知识差距,并确保不可见类的潜在空间嵌入相对于其相邻类在语义上是有意义的。我们还在图像级和数据的语义级引入了基于混合的监督,用于使用混合预测损失进行训练,这有助于在查询属于未知域时进行有效检索。这些损失被合并到SE-ResNet 50主干上以获得SnMpNet。在两个大规模数据集Sketchy Extended和DomainNet上进行了大量实验,并与最先进的数据集进行了全面的比较,证明了该模型的有效性。

英文摘要:

In this work, for the first time, we address the problem of universal cross-domain retrieval, where the test data can belong to classes or domains which are unseen during training. Due to dynamically increasing number of categories and practical constraint of training on every possible domain, which requires large amounts of data, generalizing to both unseen classes and domains is important. Towards that goal, we propose SnMpNet (Semantic Neighbourhood and Mixture Prediction Network), which incorporates two novel losses to account for the unseen classes and domains encountered during testing. Specifically, we introduce a novel Semantic Neighborhood loss to bridge the knowledge gap between seen and unseen classes and ensure that the latent space embedding of the unseen classes is semantically meaningful with respect to its neighboring classes. We also introduce a mix-up based supervision at image-level as well as semantic-level of the data for training with the Mixture Prediction loss, which helps in efficient retrieval when the query belongs to an unseen domain. These losses are incorporated on the SE-ResNet50 backbone to obtain SnMpNet. Extensive experiments on two large-scale datasets, Sketchy Extended and DomainNet, and thorough comparisons with state-of-the-art justify the effectiveness of the proposed model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑