arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.14926cs.CV
收藏

无源领域泛化的领域统一提示表示

Domain-Unified Prompt Representations for Source-Free Domain Generalization

University of Science and Technology of China(中国科学技术大学)

Hongjing Niu, Hanting Li, Feng Zhao, Bin Li

摘要:

领域泛化(DG)旨在使模型在未知领域工作,是迈向通用人工智能的必由之路。受当前DG数据集的规模和多样性的限制,现有方法难以扩展到开放世界场景中的不同领域(例如,科幻小说和像素化风格)。因此,无源领域泛化(SFDG)任务是必要的和具有挑战性的。为了解决这个问题,我们提出了一种基于大规模视觉语言预训练模型的方法(例如,CLIP),它利用了广泛的领域信息嵌入其中。所提出的方案产生不同的提示从一个域银行,其中包含了许多不同的领域比现有的DG数据集。此外,我们的方法产生域统一表示从这些提示,从而能够应付来自开放世界域的样本。在主流DG数据集,即PACS,VLCS,ECOMAHome和DomainNet上进行的大量实验表明,与需要源域数据进行训练的最新(SOTA)DG方法相比,所提出的方法具有竞争力的性能。此外,我们收集了一个由两个领域组成的小数据集,以评估所提出的方法的开放世界领域泛化能力。源代码和数据集将在https://github.com/muse1998/Source-Free-Domain-Generalization上公开提供

英文摘要:

Domain generalization (DG), aiming to make models work on unseen domains, is a surefire way toward general artificial intelligence. Limited by the scale and diversity of current DG datasets, it is difficult for existing methods to scale to diverse domains in open-world scenarios (e.g., science fiction and pixelate style). Therefore, the source-free domain generalization (SFDG) task is necessary and challenging. To address this issue, we propose an approach based on large-scale vision-language pretraining models (e.g., CLIP), which exploits the extensive domain information embedded in it. The proposed scheme generates diverse prompts from a domain bank that contains many more diverse domains than existing DG datasets. Furthermore, our method yields domain-unified representations from these prompts, thus being able to cope with samples from open-world domains. Extensive experiments on mainstream DG datasets, namely PACS, VLCS, OfficeHome, and DomainNet, show that the proposed method achieves competitive performance compared to state-of-the-art (SOTA) DG methods that require source domain data for training. Besides, we collect a small datasets consists of two domains to evaluate the open-world domain generalization ability of the proposed method. The source code and the dataset will be made publicly available at https://github.com/muse1998/Source-Free-Domain-Generalization

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑