arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.08265cs.CV
收藏

基于自然语言建模的坐标感知热红外跟踪

Coordinate-Aware Thermal Infrared Tracking Via Natural Language Modeling

School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China(电子科技大学光电科学与工程学院);Yibin Institute, University of Electronic Science and Technology of China(电子科技大学宜宾研究院);NO.208 research institute of China Ordnance Industries(中国兵器工业第208研究所)

Miao Yan, Ping Zhang, Haofei Zhang, Ruqian Hao, Juanxiu Liu, Xiaoyang Wang, Lin Liu

AI总结:

针对热红外跟踪中忽略时间和坐标信息的问题,提出基于自然语言建模的坐标感知跟踪模型NLMTrack,通过统一编码器、多级渐进融合模块和因果transformer解码器提升跟踪性能,在多个基准上达到SOTA。

摘要:

热红外(TIR)跟踪因其全天候成像能力在计算机视觉任务中至关重要。传统跟踪方法主要依赖手工特征,尽管深度学习引入了相关滤波技术,但这些技术通常受限于基础的相关操作。此外,基于transformer的方法往往忽略时间和坐标信息,这对于缺乏纹理和颜色信息的TIR跟踪至关重要。本文为解决这些问题,将自然语言建模应用于TIR跟踪,提出一种坐标感知热红外跟踪模型NLMTrack,增强了对坐标和时间信息的利用。NLMTrack应用统一特征提取和特征融合的编码器,简化了TIR跟踪流程。针对TIR图像低细节和低对比度的挑战,一方面设计了多级渐进融合模块,增强语义表示并整合多尺度特征;另一方面,解码器利用因果transformer结合TIR特征和坐标序列特征,逐步生成目标序列。此外,探索了旨在提升跟踪精度的自适应损失以及适应目标外观变化的简单模板更新策略。实验表明,NLMTrack在多个基准上实现了最先进的性能。代码已公开于\url{https://github.com/ELOESZHANG/NLMTrack}。

英文摘要:

Thermal infrared (TIR) tracking is pivotal in computer vision tasks due to its all-weather imaging capability. Traditional tracking methods predominantly rely on hand-crafted features, and while deep learning has introduced correlation filtering techniques, these are often constrained by rudimentary correlation operations. Furthermore, transformer-based approaches tend to overlook temporal and coordinate information, which is critical for TIR tracking that lacks texture and color information. In this paper, to address these issues, we apply natural language modeling to TIR tracking and propose a coordinate-aware thermal infrared tracking model called NLMTrack, which enhances the utilization of coordinate and temporal information. NLMTrack applies an encoder that unifies feature extraction and feature fusion, which simplifies the TIR tracking pipeline. To address the challenge of low detail and low contrast in TIR images, on the one hand, we design a multi-level progressive fusion module that enhances the semantic representation and incorporates multi-scale features. On the other hand, the decoder combines the TIR features and the coordinate sequence features using a causal transformer to generate the target sequence step by step. Moreover, we explore an adaptive loss aimed at elevating tracking accuracy and a simple template update strategy to accommodate the target's appearance variations. Experiments show that NLMTrack achieves state-of-the-art performance on multiple benchmarks. The Code is publicly available at \url{https://github.com/ELOESZHANG/NLMTrack}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑