arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.09896cs.CV
收藏

探索视觉基础模型在无监督域适应中的优势

Exploring the Benefits of Vision Foundation Models for Unsupervised Domain Adaptation

Eindhoven University of Technology(埃因霍温理工大学)

Brunó B. Englert, Fabrizio J. Piva, Tommie Kerssies, Daan de Geus, Gijs Dubbelman

AI总结:

本研究证明视觉基础模型与无监督域适应在语义分割中互补,能提升性能并实现显著加速,为UDA设定新标准。

摘要:

在计算机视觉中,实现跨不同数据域的鲁棒泛化仍然是一个重大挑战。这一挑战在安全关键应用中尤为重要,在这些应用中,基于深度神经网络的系统必须在训练中未见过的各种环境条件下可靠运行。我们的研究调查了视觉基础模型(VFMs)和无监督域适应(UDA)方法在语义分割任务中的泛化能力是否互补。结果表明,将VFM与UDA结合有两个主要好处:(a)它可以在保持VFM的分布外性能的同时,实现更好的UDA性能;(b)它使某些耗时的UDA组件变得多余,从而能够实现显著的推理加速。具体来说,在等效模型大小下,所得到的VFM-UDA方法相比先前非VFM的最先进方法实现了8.4倍的加速,同时在UDA设置中性能提高了+1.2 mIoU,在分布外泛化方面提高了+6.1 mIoU。此外,当我们使用参数多3.6倍的VFM时,VFM-UDA方法保持了3.3倍的加速,同时将UDA性能提高了+3.1 mIoU,分布外性能提高了+10.3 mIoU。这些结果强调了将VFM与UDA结合的巨大优势,为语义分割中的无监督域适应设定了新的标准和基线。

英文摘要:

Achieving robust generalization across diverse data domains remains a significant challenge in computer vision. This challenge is important in safety-critical applications, where deep-neural-network-based systems must perform reliably under various environmental conditions not seen during training. Our study investigates whether the generalization capabilities of Vision Foundation Models (VFMs) and Unsupervised Domain Adaptation (UDA) methods for the semantic segmentation task are complementary. Results show that combining VFMs with UDA has two main benefits: (a) it allows for better UDA performance while maintaining the out-of-distribution performance of VFMs, and (b) it makes certain time-consuming UDA components redundant, thus enabling significant inference speedups. Specifically, with equivalent model sizes, the resulting VFM-UDA method achieves an 8.4$\times$ speed increase over the prior non-VFM state of the art, while also improving performance by +1.2 mIoU in the UDA setting and by +6.1 mIoU in terms of out-of-distribution generalization. Moreover, when we use a VFM with 3.6$\times$ more parameters, the VFM-UDA approach maintains a 3.3$\times$ speed up, while improving the UDA performance by +3.1 mIoU and the out-of-distribution performance by +10.3 mIoU. These results underscore the significant benefits of combining VFMs with UDA, setting new standards and baselines for Unsupervised Domain Adaptation in semantic segmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑