从双流到单流:通过相互提示学习与知识蒸馏实现高效RGB-T跟踪
From Two-Stream to One-Stream: Efficient RGB-T Tracking via Mutual Prompt Learning and Knowledge Distillation
AI总结:
针对RGB-T跟踪中模态融合成本高的问题,提出跨模态相互提示学习的双流教师模型,并通过知识蒸馏训练快速单流学生模型,实现最高精度与三倍以上推理加速。
摘要:
由于可见光和热红外模态具有互补特性,基于可见光图像与热图像融合的目标跟踪(称为RGB-T跟踪)近年来受到研究人员越来越多的关注。如何以更低成本实现两种模态信息更全面的融合,一直是研究人员不断探索的问题。受视觉提示学习启发,我们设计了一种基于跨模态相互提示学习的新型双流RGB-T跟踪架构,并将该模型作为教师模型,通过知识蒸馏技术引导单流学生模型进行快速学习。大量实验表明,与同类RGB-T跟踪器相比,我们设计的教师模型取得了最高的精确率;而学生模型在精确率与教师模型相当的情况下,实现了比教师模型快三倍以上的推理速度。(若论文被接收,将公开代码。)
英文摘要:
Due to the complementary nature of visible light and thermal infrared modalities, object tracking based on the fusion of visible light images and thermal images (referred to as RGB-T tracking) has received increasing attention from researchers in recent years. How to achieve more comprehensive fusion of information from the two modalities at a lower cost has been an issue that researchers have been exploring. Inspired by visual prompt learning, we designed a novel two-stream RGB-T tracking architecture based on cross-modal mutual prompt learning, and used this model as a teacher to guide a one-stream student model for rapid learning through knowledge distillation techniques. Extensive experiments have shown that, compared to similar RGB-T trackers, our designed teacher model achieved the highest precision rate, while the student model, with comparable precision rate to the teacher model, realized an inference speed more than three times faster than the teacher model.(Codes will be available if accepted.)
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。