arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.00397cs.CV
收藏

COSMo:CLIP在开放集多目标域适应中的对话

COSMo: CLIP Talks on Open-Set Multi-Target Domain Adaptation

Indian Institute of Technology, Bombay(印度理工学院孟买分校);INRIA(法国国家信息与自动化研究所)

Munish Monga, Sachin Kumar Giroh, Ankit Jha, Mainak Singha, Biplab Banerjee, Jocelyn Chanussot

AI总结:

COSMo通过源域引导的提示学习,在提示空间中解决开放集多目标域适应问题,利用域特定偏置网络和独立提示处理已知与未知类别,平均提升5.1%。

摘要:

多目标域适应(MTDA)涉及从单一源域学习域不变信息,并将其应用于多个未标记的目标域。然而,现有的MTDA方法主要关注解决视觉特征中的域偏移,往往忽视语义特征,并且难以处理未知类别,从而导致所谓的开放集(OS)MTDA。虽然像CLIP这样的大规模视觉-语言基础模型显示出潜力,但其在MTDA中的潜力在很大程度上仍未得到探索。本文介绍了COSMo,一种新颖的方法,通过源域引导的提示学习来学习域无关的提示,以在提示空间中解决MTDA问题。通过利用域特定的偏置网络和针对已知与未知类别的独立提示,COSMo有效地适应了域和类别的偏移。据我们所知,COSMo是第一个解决开放集多目标域适应(OSMTDA)的方法,提供了对现实世界场景更真实的表示,并解决了开放集和多目标域适应的挑战。COSMo在三个具有挑战性的数据集:Mini-DomainNet、Office-31和Office-Home上,与其他适应于OSMTDA设置的相关域适应方法相比,平均提高了5.1%。代码可在https://github.com/munish30monga/COSMo获取。

英文摘要:

Multi-Target Domain Adaptation (MTDA) entails learning domain-invariant information from a single source domain and applying it to multiple unlabeled target domains. Yet, existing MTDA methods predominantly focus on addressing domain shifts within visual features, often overlooking semantic features and struggling to handle unknown classes, resulting in what is known as Open-Set (OS) MTDA. While large-scale vision-language foundation models like CLIP show promise, their potential for MTDA remains largely unexplored. This paper introduces COSMo, a novel method that learns domain-agnostic prompts through source domain-guided prompt learning to tackle the MTDA problem in the prompt space. By leveraging a domain-specific bias network and separate prompts for known and unknown classes, COSMo effectively adapts across domain and class shifts. To the best of our knowledge, COSMo is the first method to address Open-Set Multi-Target DA (OSMTDA), offering a more realistic representation of real-world scenarios and addressing the challenges of both open-set and multi-target DA. COSMo demonstrates an average improvement of $5.1\%$ across three challenging datasets: Mini-DomainNet, Office-31, and Office-Home, compared to other related DA methods adapted to operate within the OSMTDA setting. Code is available at: https://github.com/munish30monga/COSMo

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑