EPIPTrack:面向多目标跟踪的显式与隐式提示提示建模再思考
EPIPTrack: Rethinking Prompt Modeling with Explicit and Implicit Prompts for Multi-Object Tracking
AI总结:
针对现有多目标跟踪方法依赖静态文本描述、适应性差且易产生幻觉的问题,提出融合显式隐式提示的EPIPTrack框架,结合判别特征增强器实现动态目标建模,在多个基准上取得更优性能。
摘要:
文本描述等多模态语义线索在增强跟踪的目标感知能力方面已展现出巨大潜力。然而现有方法依赖大语言模型生成的静态文本描述,对实时目标状态变化缺乏适应性,且容易产生幻觉。为解决这些挑战,我们提出了一个统一的多模态视觉-语言跟踪框架EPIPTrack,该框架利用显式和隐式提示实现动态目标建模与语义对齐。具体而言,显式提示将空间运动信息转化为自然语言描述,以提供时空引导;隐式提示将伪词与可学习描述符相结合,构建捕捉外观属性的个性化知识表示。两种提示均通过CLIP文本编码器进行动态调整,以响应目标状态的变化。此外,我们设计了一个判别特征增强器(Discriminative Feature Augmentor)来增强视觉表征与跨模态表征。在MOT17、MOT20和DanceTrack上开展的大量实验表明,EPIPTrack在多种场景下的表现优于现有跟踪器,具备鲁棒的适应性与更优的性能。
英文摘要:
Multimodal semantic cues, such as textual descriptions, have shown strong potential in enhancing target perception for tracking. However, existing methods rely on static textual descriptions from large language models, which lack adaptability to real-time target state changes and prone to hallucinations. To address these challenges, we propose a unified multimodal vision-language tracking framework, named EPIPTrack, which leverages explicit and implicit prompts for dynamic target modeling and semantic alignment. Specifically, explicit prompts transform spatial motion information into natural language descriptions to provide spatiotemporal guidance. Implicit prompts combine pseudo-words with learnable descriptors to construct individualized knowledge representations capturing appearance attributes. Both prompts undergo dynamic adjustment via the CLIP text encoder to respond to changes in target state. Furthermore, we design a Discriminative Feature Augmentor to enhance visual and cross-modal representations. Extensive experiments on MOT17, MOT20, and DanceTrack demonstrate that EPIPTrack outperforms existing trackers in diverse scenarios, exhibiting robust adaptability and superior performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。