发表机构
College of Computer Science and Electronic Engineering, Hunan University; Huaihua University（湖南大学计算机与电子工程学院; 怀化学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对无源全自由域适应的双语义漂移问题，提出DSSG框架及带原型锚定校准的DSSG-PAC，在多基准上性能优于SOTA且适配时间降低18.9%。
AI 中文摘要
无源全自由域适应（SFF-DA）已成为一种无需访问任何源数据或特定任务源模型即可适配视觉语言模型（VLMs）的战略范式。然而，我们发现阻碍该过程的关键双语义漂移：由固定类嵌入的刚性导致的静态漂移，以及由生成描述的发散导致的动态漂移，造成严重的语义不对齐，加剧了稳定性-可塑性困境。为解决该问题，我们提出DSSG（双流语义引导），一种协调细粒度可塑性与全局稳定性的端到端框架。核心贡献是双语义引导（DSG）模块，其集成用于特定领域知识的描述流与用于锚定全局类别一致性的类锚流。此外，引入动态跨模态知识蒸馏（CMKD）模块，利用演化的教师分布校准师生一致性。在DSSG基础上，我们进一步引入原型锚定校准（PAC），得到DSSG-PAC，其定期校准原型锚并缓存至下一次校准，该设计减少了文本侧冗余计算，同时保留类引导对演化文本空间的适应性。我们还建立SFF-DA风险边界，将学生风险与语义教师质量及师生差异关联。大量实验表明，DSSG在多个基准上始终优于当前最先进方法，而DSSG-PAC在保留适配性能的同时，总适配时间降低18.9%。代码可在该https URL获取。
英文摘要
Source-Fully-Free Domain Adaptation (SFF-DA) has emerged as a strategic paradigm to adapt Vision-Language Models (VLMs) without any access to source data or task-specific source models. However, we identify a critical Dual Semantic Drift that hinders this process: static drift arising from the rigidity of fixed class embeddings, and dynamic drift stemming from the divergence of generated captions, causing severe semantic misalignment that intensifies the stability-plasticity dilemma. To address this, we propose DSSG (Dual-Stream Semantic Guidance), an end-to-end framework that reconciles fine-grained plasticity with global stability. Our core contribution is the Dual Semantic Guidance (DSG) module, which integrates a caption stream for domain-specific knowledge with a class-anchor stream to anchor global categorical consistency. Furthermore, a Dynamic Cross-Modal Knowledge Distillation (CMKD) module is introduced to leverage the evolving teacher distribution for calibrating teacher-student consistency. Building upon DSSG, we further introduce Prototype Anchor Calibration (PAC), yielding DSSG-PAC, which periodically calibrates prototype anchors and caches them until the next calibration. This design reduces redundant text-side computation while preserving the adaptability of class guidance to the evolving text space. We further establish SFF-DA risk bounds that relate student risk to semantic-teacher quality and teacher--student discrepancy. Extensive experiments demonstrate that DSSG consistently outperforms current state-of-the-art methods across multiple benchmarks, while DSSG-PAC largely preserves its adaptation performance with 18.9% lower total adaptation time. The code is available at https://github.com/mrmenand/DSSG.
