arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.18481cs.CVcs.LG
收藏

T-QPM:面向开放世界视觉语言模型的时间分布外检测与域泛化

T-QPM: Enabling Temporal Out-Of-Distribution Detection and Domain Generalization for Vision-Language Models in Open-World

San Diego State University(圣地亚哥州立大学)

Aditi Naiknaware, Salimeh Sekeh

AI总结:

提出T-QPM框架,通过时间四元模式匹配和轻量级融合权重学习,增强视觉语言模型在动态环境下的分布外检测和协变量偏移鲁棒性。

摘要:

分布外(OOD)检测在开放世界学习中仍然是一个关键挑战,模型必须适应不断变化的数据分布。虽然最近的视觉语言模型(VLM)如CLIP通过双模式匹配(DPM)实现了多模态OOD检测,但现有方法通常存在两个主要缺点:(1)它们依赖固定的融合规则并假设静态环境,在时间漂移下失效;(2)缺乏对协变量偏移输入的鲁棒性。在本文中,我们提出了一种新颖的两步框架,以增强动态环境下的OOD检测和协变量分布偏移鲁棒性。我们将双模式机制扩展为时间四元模式匹配(T-QPM)。首先,通过将OOD图像与文本描述配对,我们引入了ID和OOD信号之间的跨模态一致性模式,通过联合图像-文本推理细化决策边界。其次,我们通过学习轻量级融合权重来最优地结合语义匹配和视觉典型性,从而解决时间分布偏移。为了确保稳定性,我们基于平均阈值置信度(ATC)强制执行显式正则化,防止随着分布演化而性能下降。在时间分区基准上的实验表明,我们的方法显著优于静态基线,为非平稳环境下的多模态OOD检测提供了一个鲁棒的、时间一致的框架。

英文摘要:

Out-of-distribution (OOD) detection remains a critical challenge in open-world learning, where models must adapt to evolving data distributions. While recent vision-language models (VLMS) like CLIP enable multimodal OOD detection through Dual-Pattern Matching (DPM), existing methods typically suffer from two major shortcomings: (1) They rely on fixed fusion rules and assume static environments, failing under temporal drift; and (2) they lack robustness against covariate shifted inputs. In this paper, we propose a novel two-step framework to enhance OOD detection and covariate distribution shift robustness in dynamic settings. We extend the dual-pattern regime into Temporal Quadruple-Pattern Matching (T-QPM). First, by pairing OOD images with text descriptions, we introduce cross-modal consistency patterns between ID and OOD signals, refining the decision boundary through joint image-text reasoning. Second, we address temporal distribution shifts by learning lightweight fusion weights to optimally combine semantic matching and visual typicality. To ensure stability, we enforce explicit regularization based on Average Thresholded Confidence (ATC), preventing performance degradation as distributions evolve. Experiments on temporally partitioned benchmarks demonstrate that our approach significantly outperforms static baselines, offering a robust, temporally-consistent framework for multimodal OOD detection in non-stationary environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑