arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2308.13331cs.CV
收藏

用于域泛化语义分割的重参数化视觉Transformer(ReVT)

A Re-Parameterized Vision Transformer (ReVT) for Domain-Generalized Semantic Segmentation

Technische Universitat Braunschweig(布伦瑞克工业大学)

Jan-Aike Termöhlen, Timo Bartels, Tim Fingscheidt

AI总结:

针对语义分割模型在未见域性能下降的问题,提出基于重参数化视觉Transformer(ReVT)的增强驱动域泛化方法,训练后多模型权重平均,在基准数据集上取得更优mIoU,且参数量少、帧率高、推理无额外复杂度。

摘要:

语义分割任务要求模型为图像的每个像素分配语义标签。然而,当这类模型部署在与训练域数据分布不同的未见域中时,性能会下降。我们提出了一种新的以增强为驱动的语义分割域泛化方法,该方法采用重参数化视觉Transformer(ReVT),并在训练后对多个模型的权重进行平均。我们在多个基准数据集上评估了该方法,在常用基准数据集上,小模型取得了47.3%的最优mIoU性能(现有最优水平为46.3%),中等规模模型取得了50.1%的最优mIoU性能(现有最优水平为47.8%)。同时,我们的方法参数量更少,帧率比现有最优方法更高。该方法还易于实现,且与网络集成不同,推理阶段不会增加任何计算复杂度。

英文摘要:

The task of semantic segmentation requires a model to assign semantic labels to each pixel of an image. However, the performance of such models degrades when deployed in an unseen domain with different data distributions compared to the training domain. We present a new augmentation-driven approach to domain generalization for semantic segmentation using a re-parameterized vision transformer (ReVT) with weight averaging of multiple models after training. We evaluate our approach on several benchmark datasets and achieve state-of-the-art mIoU performance of 47.3% (prior art: 46.3%) for small models and of 50.1% (prior art: 47.8%) for midsized models on commonly used benchmark datasets. At the same time, our method requires fewer parameters and reaches a higher frame rate than the best prior art. It is also easy to implement and, unlike network ensembles, does not add any computational complexity during inference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑