何时适应：开放词汇分割中用于高效免训练适应的多信号域偏移检测
When to Adapt: Multi-Signal Domain Shift Detection for Efficient Training-Free Adaptation in Open-Vocabulary Segmentation
- KTH Royal Institute of Technology（瑞典皇家理工学院）
- Ericsson AB（爱立信公司）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对开放词汇分割中免训练连续测试时适应，提出多信号域偏移检测方法，利用时间一致性监控视觉变化、适配器不匹配和语义漂移，仅在需要时触发适应，在保持精度的同时减少适应次数，适用于长期机器人部署。
AI中文摘要：
鲁棒且可靠的感知对于在现实环境中运行的自主机器人至关重要，尤其是在环境条件可能随时间显著变化的长期任务中。尽管视觉基础模型（VFMs）的最新进展改善了开放词汇语义分割，但这些模型仍可能遭受域偏移的影响，如果它们未适应当前环境，性能可能会显著下降。免训练域适应是一种相关的适应范式，包括使用轻量级适配器在线调整模型。近期方法在逐帧基础上应用此方法，这对于资源受限的机器人硬件部署而言不切实际。为解决此问题，我们提出了一种用于开放词汇分割中免训练连续测试时适应（TF-CTTA）的多信号域偏移检测方法。我们的方法利用连续帧之间的时间一致性，通过监控并结合域偏移的互补方面（视觉变化、适配器不匹配和语义漂移），仅在需要时触发适应。我们在包含室内和室外环境并使用真实机器人数据的基准上验证了我们的方法。我们证明，我们的方法在保持分割精度的同时大幅减少适应次数，使免训练适应对于长期、现实世界的机器人部署变得实用且可行。
英文摘要：
Robust and reliable perception is essential for autonomous robots operating in real-world environments, particularly in long-term missions where environmental conditions may change significantly over time. Although recent advances in Visual Foundation Models (VFMs) have improved open-vocabulary semantic segmentation, these models can still suffer from domain shift, which can significantly degrade performance if they are not adapted to the current environment. Training-free domain adaptation is a relevant paradigm for adaptation, consisting of adjusting the model online using lightweight adapters. Recent approaches apply this on a per-frame basis, which is impractical for deployments on resource-constrained robotic hardware. To tackle this, we propose a multi-signal domain shift detection method for training-free continual test-time adaptation (TF-CTTA) in open-vocabulary segmentation. Our method leverages temporal coherence across consecutive frames by monitoring and combining complementary aspects of domain shift (visual change, adapter mismatch, and semantic drift) to trigger adaptation only when needed. We validate our approach on a benchmark including indoor and outdoor environments and using real robotic data. We demonstrate that our approach maintains segmentation accuracy while substantially reducing adaptations, making training-free adaptation practical and feasible for long-term, real-world robotic deployments.
