arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.02899cs.AI
收藏

面向无监督域适应的域无关互提示方法

Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation

University of Electronic Science and Technology of China(电子科技大学);University of Technology Sydney(悉尼科技大学);Tongji University(同济大学)

Zhekai Du, Xinyao Li, Fengling Li, Ke Lu, Lei Zhu, Jingjing Li

AI总结:

针对传统无监督域适应忽略数据语义、跨域迁移受限的问题,提出域无关互提示方法DAMP,通过视觉与文本提示的互学习挖掘域不变语义,在三个基准上取得优于现有最优方法的效果。

摘要:

传统无监督域适应(Unsupervised Domain Adaptation, UDA)致力于最小化域间分布差异,却忽略了对数据中丰富语义信息的利用,且难以应对复杂的域偏移问题。一种颇具前景的技术是借助大规模预训练视觉-语言模型的知识,实现更具引导性的域适应。尽管已有相关尝试,但现有方法通常分别学习文本提示以嵌入源域和目标域的域语义,并在各自域内执行分类,这限制了跨域知识迁移。此外,仅对语言分支施加提示,缺乏动态适配两种模态的灵活性。为弥补这一不足,我们提出域无关互提示(Domain-Agnostic Mutual Prompting, DAMP)方法,通过互对齐视觉嵌入与文本嵌入来挖掘域不变语义。具体而言,该方法利用图像上下文信息,以域无关、实例条件化的方式提示语言分支;同时基于域无关文本提示施加视觉提示,以提取域不变视觉嵌入。两个分支的提示通过交叉注意力模块互学习,并由语义一致性损失和实例判别对比损失进行正则化约束。在三个UDA基准数据集上的实验表明,DAMP的性能优于现有最优方法。

英文摘要:

Conventional Unsupervised Domain Adaptation (UDA) strives to minimize distribution discrepancy between domains, which neglects to harness rich semantics from data and struggles to handle complex domain shifts. A promising technique is to leverage the knowledge of large-scale pre-trained vision-language models for more guided adaptation. Despite some endeavors, current methods often learn textual prompts to embed domain semantics for source and target domains separately and perform classification within each domain, limiting cross-domain knowledge transfer. Moreover, prompting only the language branch lacks flexibility to adapt both modalities dynamically. To bridge this gap, we propose Domain-Agnostic Mutual Prompting (DAMP) to exploit domain-invariant semantics by mutually aligning visual and textual embeddings. Specifically, the image contextual information is utilized to prompt the language branch in a domain-agnostic and instance-conditioned way. Meanwhile, visual prompts are imposed based on the domain-agnostic textual prompt to elicit domain-invariant visual embeddings. These two branches of prompts are learned mutually with a cross-attention module and regularized with a semantic-consistency loss and an instance-discrimination contrastive loss. Experiments on three UDA benchmarks demonstrate the superiority of DAMP over state-of-the-art approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑