MambaST：一种用于高效行人检测的即插即用跨光谱时空融合器
MambaST: A Plug-and-Play Cross-Spectral Spatial-Temporal Fuser for Efficient Pedestrian Detection
AI总结：
本文提出即插即用的跨光谱时空融合流程MambaST，利用状态空间模型Mamba和多头分层分块聚合结构MHHPA融合RGB与热成像信息，实现高效且鲁棒的行人检测，尤其在小尺度行人检测上表现优越。
摘要：
本文提出了MambaST，一种用于高效行人检测的即插即用跨光谱时空融合流程。自动驾驶应用中的行人检测面临若干挑战。首先，在黑暗或低光照条件下，使用RGB相机难以实现准确检测。必须开发跨光谱系统，以整合来自多个传感器模态（如热成像相机和可见光相机）的互补信息，从而提高检测的鲁棒性。其次，行人检测模型对延迟敏感。对于自动驾驶等实时应用而言，非常需要参数更少、高效且易于扩展的检测模型。第三，行人视频数据提供了行人运动的时空相关性。同时利用时间信息和空间信息有助于增强行人检测。本工作利用了状态空间模型（Mamba）的最新进展，并提出了一种新颖的多头分层分块与聚合（MHHPA）结构，以从RGB和热成像图像中提取细粒度和粗粒度信息。实验结果表明，所提出的MHHPA是跨光谱行人检测中Transformer模型的一种有效且高效的替代方案。我们提出的模型在小尺度行人检测上也取得了优越的性能。代码可在https://github.com/XiangboGaoBarry/MambaST获取。
英文摘要：
This paper proposes MambaST, a plug-and-play cross-spectral spatial-temporal fusion pipeline for efficient pedestrian detection. Several challenges exist for pedestrian detection in autonomous driving applications. First, it is difficult to perform accurate detection using RGB cameras under dark or low-light conditions. Cross-spectral systems must be developed to integrate complementary information from multiple sensor modalities, such as thermal and visible cameras, to improve the robustness of the detections. Second, pedestrian detection models are latency-sensitive. Efficient and easy-to-scale detection models with fewer parameters are highly desirable for real-time applications such as autonomous driving. Third, pedestrian video data provides spatial-temporal correlations of pedestrian movement. It is beneficial to incorporate temporal as well as spatial information to enhance pedestrian detection. This work leverages recent advances in the state space model (Mamba) and proposes a novel Multi-head Hierarchical Patching and Aggregation (MHHPA) structure to extract both fine-grained and coarse-grained information from both RGB and thermal imagery. Experimental results show that the proposed MHHPA is an effective and efficient alternative to a Transformer model for cross-spectral pedestrian detection. Our proposed model also achieves superior performance on small-scale pedestrian detection. The code is available at https://github.com/XiangboGaoBarry/MambaST}{https://github.com/XiangboGaoBarry/MambaST.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
