VLMs 与 UDA 的碰撞：通过无监督域适应提升开放词汇分割的可迁移性
VLMs meet UDA: Boosting Transferability of Open Vocabulary Segmentation with Unsupervised Domain Adaptation
AI总结：
针对分割模型类别受限及现有方法的不足，本文提出UDA-FROVSS框架，结合VLM与UDA策略，通过多尺度上下文、提示增强等提升细粒度分割能力，实现无需共享类别的有效跨域开放词汇分割。
摘要：
分割模型通常受限于训练时定义的类别。为解决这一问题，研究人员探索了两种独立的方法：适配视觉-语言模型（Vision-Language Models, VLMs）以及利用合成数据。然而，VLMs 往往在粒度处理上存在困难，无法区分细粒度概念，而基于合成数据的方法仍受限于可用数据集的范围。\n 本文提出将视觉-语言推理与无监督域适应（Unsupervised Domain Adaptation, UDA）的关键策略相结合，以提升跨不同域的分割准确率。首先，我们在提出的 Foundational-Retaining Open Vocabulary Semantic Segmentation（FROVSS，基础保留型开放词汇语义分割）框架中，通过多尺度上下文数据、带提示增强的鲁棒文本嵌入以及逐层微调，提升了 VLMs 的细粒度分割能力。接着，我们将这些增强功能整合到 UDA 框架中，采用蒸馏法稳定训练，并使用跨域混合采样在不损害泛化性的前提下提升适应能力。最终得到的 UDA-FROVSS 框架是首个无需共享类别即可实现有效跨域适配的 UDA 方法。
英文摘要：
Segmentation models are typically constrained by the categories defined during training. To address this, researchers have explored two independent approaches: adapting Vision-Language Models (VLMs) and leveraging synthetic data. However, VLMs often struggle with granularity, failing to disentangle fine-grained concepts, while synthetic data-based methods remain limited by the scope of available datasets. This paper proposes enhancing segmentation accuracy across diverse domains by integrating Vision-Language reasoning with key strategies for Unsupervised Domain Adaptation (UDA). First, we improve the fine-grained segmentation capabilities of VLMs through multi-scale contextual data, robust text embeddings with prompt augmentation, and layer-wise fine-tuning in our proposed Foundational-Retaining Open Vocabulary Semantic Segmentation (FROVSS) framework. Next, we incorporate these enhancements into a UDA framework by employing distillation to stabilize training and cross-domain mixed sampling to boost adaptability without compromising generalization. The resulting UDA-FROVSS framework is the first UDA approach to effectively adapt across domains without requiring shared categories.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
