arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2208.09698cs.CVcs.AIcs.LG
收藏

学习与参与:艺术与素描的广义嵌入学习

Fuse and Attend: Generalized Embedding Learning for Art and Sketches

Myntra(Myntra公司)

Ujjal Kr Dutta

摘要:

虽然深度嵌入学习方法在多个计算机视觉任务中取得了广泛的成功,但用于表示自然图像的最先进方法不一定在其他领域的图像上表现良好,例如绘画,卡通和素描。这是因为与自然图像相比,这些领域的数据分布发生了巨大的变化。像sketch这样的域通常包含稀疏的信息像素。然而,识别这些领域中的对象是至关重要的,因为多个相关应用程序利用这些数据,例如,草图到图像检索。因此,实现一个可以在多个领域表现良好的嵌入式学习模型不仅具有挑战性,而且在计算机视觉中发挥着关键作用。为此,在本文中,我们提出了一种新的嵌入式学习方法,其目标是在不同的领域进行推广。在训练过程中,给定来自某个领域的查询图像,我们采用门控融合和注意力来生成一个正面示例,该示例包含查询对象类别(来自多个领域)的语义的广泛概念。通过对比学习,我们提取查询和正的嵌入,以便学习跨域鲁棒的表示。与此同时,为了让模型能够区分来自不同语义类别(跨领域)的示例,我们还维护了一个负嵌入池(来自不同类别)。我们使用DomainBed框架展示了我们的方法在流行的PACS(照片,艺术绘画,卡通和素描)数据集上的强大功能。

英文摘要:

While deep Embedding Learning approaches have witnessed widespread success in multiple computer vision tasks, the state-of-the-art methods for representing natural images need not necessarily perform well on images from other domains, such as paintings, cartoons, and sketch. This is because of the huge shift in the distribution of data from across these domains, as compared to natural images. Domains like sketch often contain sparse informative pixels. However, recognizing objects in such domains is crucial, given multiple relevant applications leveraging such data, for instance, sketch to image retrieval. Thus, achieving an Embedding Learning model that could perform well across multiple domains is not only challenging, but plays a pivotal role in computer vision. To this end, in this paper, we propose a novel Embedding Learning approach with the goal of generalizing across different domains. During training, given a query image from a domain, we employ gated fusion and attention to generate a positive example, which carries a broad notion of the semantics of the query object category (from across multiple domains). By virtue of Contrastive Learning, we pull the embeddings of the query and positive, in order to learn a representation which is robust across domains. At the same time, to teach the model to be discriminative against examples from different semantic categories (across domains), we also maintain a pool of negative embeddings (from different categories). We show the prowess of our method using the DomainBed framework, on the popular PACS (Photo, Art painting, Cartoon, and Sketch) dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑