ED-SAM：一种面向视觉-语言基础模型领域泛化的高效扩散采样方法
ED-SAM: An Efficient Diffusion Sampling Approach to Domain Generalization in Vision-Language Foundation Models
AI总结：
本文提出ED-SAM方法，通过引入Transport Transformation生成对抗样本，有效提升了视觉-语言基础模型对未知数据分布的泛化能力，并在多个数据集上取得了最先进性能。
摘要：
视觉-语言基础模型近期在各种感知学习任务中展现出卓越的性能。这种卓越性能主要依赖于大规模预训练数据集和不同的数据增强技术。然而，视觉-语言基础模型的领域泛化问题亟待解决。该问题限制了视觉-语言基础模型对未知数据分布的泛化能力。在本文中，我们引入了一种新颖、简单但高效的面向领域泛化的扩散采样方法（ED-SAM），以提升视觉-语言基础模型的泛化能力。我们在本工作中的理论分析揭示了扩散模型与视觉-语言基础模型领域泛化之间的关键作用与关联。随后，基于这一深刻分析，我们为扩散采样方法引入了一种新颖、简单且有效的Transport Transformation。它能有效地生成对抗样本，从而提升基础模型对未知数据分布的泛化能力。在不同规模的视觉-语言预训练数据集（包括CC3M、CC12M和LAION400M）上的实验结果一致表明，与其他最新方法相比，所提出的ED-SAM方法具有最先进的性能和可扩展性。
英文摘要：
The Vision-Language Foundation Model has recently shown outstanding performance in various perception learning tasks. The outstanding performance of the vision-language model mainly relies on large-scale pre-training datasets and different data augmentation techniques. However, the domain generalization problem of the vision-language foundation model needs to be addressed. This problem has limited the generalizability of the vision-language foundation model to unknown data distributions. In this paper, we introduce a new simple but efficient Diffusion Sampling approach to Domain Generalization (ED-SAM) to improve the generalizability of the vision-language foundation model. Our theoretical analysis in this work reveals the critical role and relation of the diffusion model to domain generalization in the vision-language foundation model. Then, based on the insightful analysis, we introduce a new simple yet effective Transport Transformation to diffusion sampling method. It can effectively generate adversarial samples to improve the generalizability of the foundation model against unknown data distributions. The experimental results on different scales of vision-language pre-training datasets, including CC3M, CC12M, and LAION400M, have consistently shown State-of-the-Art performance and scalability of the proposed ED-SAM approach compared to the other recent methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
