GLAD: 生成式语言辅助低语义模板视觉跟踪
GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates
AI总结：
GLAD通过生成式多模态融合提升低语义模板视觉跟踪性能，实现更高效的多模态特征融合与语义增强。
摘要：
视觉-语言跟踪在许多场景中获得了越来越多的关注。该任务同时处理视觉和语言信息以在视频中定位物体。尽管其应用日益广泛，但视觉-语言跟踪方法的发展仍处于初级阶段。当前的视觉-语言跟踪器通常使用Transformer架构来交互整合模板、搜索和文本特征。然而，关于低语义图像的持续挑战，包括普遍的图像模糊、低分辨率等，可能会通过降质的跨模态理解来损害模型性能。为了解决这个问题，语言辅助通常用于处理低语义图像带来的障碍。然而，由于现有文本和视觉特征之间的差距，直接拼接和融合这些特征可能效果有限。为了解决这些挑战，我们引入了一种开创性的生成式语言辅助跟踪模型GLAD，该模型利用扩散模型对文本描述和模板图像进行生成式多模态融合，以增强语言和图像之间的兼容性，并增强模板图像的语义信息。我们的方法在现有融合范式上展示了显著的改进。模糊和语义模糊的模板图像可以通过生成式融合范式得到恢复，以提高多模态特征。实验表明，我们的方法在多个基准上建立了新的最先进的水平，并实现了令人印象深刻的推理速度。代码和模型将在：https://github.com/Confetti-lxy/GLAD上发布。
英文摘要：
Vision-language tracking has gained increasing attention in many scenarios. This task simultaneously deals with visual and linguistic information to localize objects in videos. Despite its growing utility, the development of vision-language tracking methods remains in its early stage. Current vision-language trackers usually employ Transformer architectures for interactive integration of template, search, and text features. However, persistent challenges about low-semantic images including prevalent image blurriness, low resolution and so on, may compromise model performance through degraded cross-modal understanding. To solve this problem, language assistance is usually used to deal with the obstacles posed by low-semantic images. However, due to the existing gap between current textual and visual features, direct concatenation and fusion of these features may have limited effectiveness. To address these challenges, we introduce a pioneering Generative Language-AssisteD tracking model, GLAD, which utilizes diffusion models for the generative multi-modal fusion of text description and template image to bolster compatibility between language and image and enhance template image semantic information. Our approach demonstrates notable improvements over the existing fusion paradigms. Blurry and semantically ambiguous template images can be restored to improve multi-modal features in the generative fusion paradigm. Experiments show that our method establishes a new state-of-the-art on multiple benchmarks and achieves an impressive inference speed. The code and models will be released at: https://github.com/Confetti-lxy/GLAD
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
