Multispectral images (e.g. visible and infrared) may be particularly useful when detecting objects with the same model in different environments (e.g. day/night outdoor scenes). To effectively use the different spectra, the main technical problem resides in the information fusion process. In this paper, we propose a new halfway feature fusion method for neural networks that leverages the complementary/consistency balance existing in multispectral features by adding to the network architecture, a particular module that cyclically fuses and refines each spectral feature. We evaluate the effectiveness of our fusion method on two challenging multispectral datasets for object detection. Our results show that implementing our Cyclic Fuse-and-Refine module in any network improves the performance on both datasets compared to other state-of-the-art multispectral object detection methods.
Dual Sparse Aggregation Transformer for Multispectral Object Detection
Northwestern Polytechnical University(西北工业大学);School of Computer Science, Northwestern Polytechnical University(西北工业大学计算机学院);School of Cybersecurity, Northwestern Polytechnical University(西北工业大学网络空间安全学院);Shaanxi Provincial Key Laboratory of Speech and Image Information Processing(陕西省语音与图像信息处理重点实验室);National Engineering Laboratory for Integrated Aerospace-Ground-Ocean Big Data Application Technology(空天地海一体化大数据应用技术国家工程实验室)
Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection
School of Electrical and Information Engineering, Jiangsu University(江苏大学电气与信息工程学院);School of Computer Science and Engineering, Jiangsu University of Science and Technology(江苏科技大学计算机科学与工程学院)
MO R-CNN: Multispectral Oriented R-CNN for Object Detection in Remote Sensing Image
College of Electronics and Information Engineering, Sichuan University(四川大学电子信息工程学院);School of Mechanical Engineering and Automation, Fuzhou University(福州大学机械工程与自动化学院);Yunnan Key Laboratory of Software Engineering, Yunnan University(云南软件工程重点实验室)
针对多光谱定向目标检测中模态差异与计算开销问题,提出轻量级框架 MO R-CNN,通过异构特征提取、单模态监督和条件标签融合提升检测性能。
TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection
College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息与电子工程学院);Jinhua Institute of Zhejiang University(浙江大学金华研究院);School of Computer Science, University of Nottingham Ningbo China(宁波诺丁汉大学计算机学院)
School of Electrical and Information Engineering, Jiangsu University, Zhenjiang, 212013, China(江苏大学电气与信息工程学院);School of Computer Science and Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, China(江苏科技大学计算机科学与工程学院);School of Automation, Southeast University, Nanjing, 210096, China(东南大学自动化学院);Department of Computer Science and Engineering, University of North Texas, Denton, TX 76207, USA(北卡罗来纳州立大学计算机科学与工程系);School of Computing, Nanjing Normal University, Nanjing, 210046, Jiangsu(南京师范大学计算机学院)
Selective Structured State Space for Multispectral-fused Small Target Detection
Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学);National Space Science Center, Chinese Academy of Sciences(中国科学院国家空间科学中心);School of Computer Science and Technology, University of Chinese Academy of Sciences(中国科学院大学计算机科学与技术学院);School of Astronomy and Space Science, University of Chinese Academy of Sciences(中国科学院大学天文与空间科学学院);School of Computing, National University of Singapore(新加坡国立大学计算机学院)
DPDETR: Decoupled Position Detection Transformer for Infrared-Visible Object Detection
School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications(通信与信息工程学院,重庆邮电大学);School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University(智能系统工程学院,中山大学深圳校区);School of Mathematics and Statistics, Xi’an Jiaotong University(数学与统计学院,西安交通大学)
WCCNet: Wavelet-context Cooperative Network for Efficient Multispectral Pedestrian Detection
the College of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院);the College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息科学与电子工程学院)
FuseNet: Self-Supervised Dual-Path Network for Medical Image Segmentation
Iran University of Science and Technology(伊朗科技大学);University of Regensburg(雷根斯堡大学);RWTH Aachen University(亚琛工业大学);Northwestern University(西北大学);Fraunhofer Institute for Digital Medicine MEVIS(弗劳恩霍夫数字医学MEVIS研究所)
An Attention-Fused Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);University of Chinese Academy of Sciences(中国科学院大学);The University of New South Wales(新南威尔士大学);Australian Defence Force Academy(澳大利亚国防学院)
Implicit Neural Feature Fusion Function for Multispectral and Hyperspectral Image Fusion
University of Electronic Science and Technology of China(电子科技大学);Institute of Methodologies for Environmental Analysis (CNR-IMAA)(环境分析研究所(CNR-IMAA));National Biodiversity Future Center(国家生物多样性未来中心)
Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Renmin University of China(中国人民大学);CEBSIT, Chinese Academy of Sciences(中国科学院CEBSIT)
MS-DETR: Multispectral Pedestrian Detection Transformer with Loosely Coupled Fusion and Modality-Balanced Optimization
School of Computer Science, Northwestern Polytechnical University(西北工业大学计算机学院);Research Development Institute of Northwestern Polytechnical University in Shenzhen(西北工业大学深圳研究院)
Unsupervised Hyperspectral and Multispectral Image Blind Fusion Based on Deep Tucker Decomposition Network with Spatial-Spectral Manifold Learning
School of Computer Science and Engineering, Nanjing University of Science and Technology (NJUST)(南京理工大学计算机科学与工程学院);Geological Exploration Technology Institute of Jiangsu Province(江苏省地质勘查技术院);Jiangsu Province Engineering Research Center of Airborne Detecting and Intelligent Perceptive Technology(江苏省机载探测与智能感知技术工程研究中心)
Multispectral Video Fusion for Non-contact Monitoring of Respiratory Rate and Apnea
ETH Zurich(苏黎世联邦理工学院)
在许多临床应用中需要连续监测呼吸活动以检测呼吸事件。呼吸的非接触式监测可以通过近红外和远红外光谱相机来实现。然而,目前的技术不足以用于临床应用。例如,他们无法估计呼吸暂停期间的准确呼吸率(RR)。我们提出了一种新的算法的基础上,多光谱数据融合的目的是在呼吸暂停估计RR。该算法独立解决RR估计和呼吸暂停检测任务。从多个源提取呼吸信息,并将其馈送到RR估计器和呼吸暂停检测器中,其结果被融合到最终的呼吸活动估计中。我们回顾性评估了该系统的数据来自30名健康成年人谁执行不同的控制呼吸任务,而仰卧在一个黑暗的房间里,再现中枢和阻塞性呼吸暂停事件。结合多光谱相机的多个呼吸信息提高了RR估计的均方根误差(RMSE)准确性,从高达4.64单光谱数据下降到1.60次呼吸/min。分类阻塞性(0.75至0.86)和中枢性呼吸暂停(0.75至0.93)的中位数F1评分也有所提高。此外,呼吸暂停检测的独立考虑导致更稳健的系统(RMSE为4.44 vs. 7.96次呼吸/min)。我们的研究结果可能代表了在医疗应用中使用相机进行生命体征监测的一步。