Across a wide range of applications, from autonomous vehicles to medical imaging, multi-spectral images provide an opportunity to extract additional information not present in color images. One of the most important steps in making this information readily available is the accurate estimation of dense correspondences between different spectra.
Due to the nature of cross-spectral images, most correspondence solving techniques for the visual domain are simply not applicable. Furthermore, most cross-spectral techniques utilize spectra-specific characteristics to perform the alignment. In this work, we aim to address the dense correspondence estimation problem in a way that generalizes to more than one spectrum. We do this by introducing a novel cycle-consistency metric that allows us to self-supervise. This, combined with our spectra-agnostic loss functions, allows us to train the same network across multiple spectra.
We demonstrate our approach on the challenging task of dense RGB-FIR correspondence estimation. We also show the performance of our unmodified network on the cases of RGB-NIR and RGB-RGB, where we achieve higher accuracy than similar self-supervised approaches. Our work shows that cross-spectral correspondence estimation can be solved in a common framework that learns to generalize alignment across spectra.
Optimizing Multispectral Object Detection: A Bag of Tricks and Comprehensive Benchmarks
Beijing Forestry University(北京林业大学);Hong Kong University of Science and Technology(香港科技大学);State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)(多模态人工智能系统国家重点实验室);National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);Sun Yat-sen University(中山大学)
Towards UAV Detection in the Real World: A New Multispectral Dataset UAVNet-MS and a New Method
College of Electronic Science and Technology, National University of Defense Technology(电子科学与技术学院,国防科技大学);Aviation University of Air Force(空军航空大学);Sun Yat-sen University(中山大学)
Multispectral Detection Transformer with Infrared-Centric Feature Fusion
Artificial Intelligence Graduate School, Gwangju Institute of Science and Technology (GIST)(人工智能研究生院,全州科学技术院(GIST));School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST)(电气工程与计算机科学学院,全州科学技术院(GIST))
Bayesian Fully-Connected Tensor Network for Hyperspectral-Multispectral Image Fusion
School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);School of Computer Science and Artificial Intelligence, Zhengzhou University(郑州大学计算机科学与人工智能学院);Department of Computer Science, St. Francis Xavier University(圣弗朗西斯大学计算机科学系)
SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensing
School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Artificial Intelligence and Automation, Wuhan University of Science and Technology(武汉科技大学人工智能与自动化学院);School of Electronic Information, Wuhan University of Science and Technology(武汉科技大学电子信息学院);School of Computer Science, Wuhan University(武汉大学计算机学院)
Multispectral image denoising with optimized vector non-local mean filter
CSE Department, College of Engineering, Qatar University, P.O. Box 2713, Doha-Qatar(卡塔尔大学工程学院计算机科学与工程系);LE2i Lab, UMR CNRS 6306, University of Burgundy, BP 47870, 21078, Dijon-France(布尔格斯大学);Department of Computer Science, University of Biskra, 07000, Algeria(阿尔及利亚比斯克拉大学计算机科学系)
Event Spectroscopy: Event-based Multispectral and Depth Sensing using Structured Light
Environmental Robotics Laboratory, Dep. of Environmental Systems Science, ETH Zurich(环境机器人实验室,环境系统科学系,苏黎世联邦理工学院);Swiss Federal Institute for Forest, Snow and Landscape Research (WSL)(瑞士森林、雪和景观研究联邦研究所);Robotics and Perception Group, University of Zurich(机器人与感知小组,苏黎世大学)
Spectral and Spatial Graph Learning for Multispectral Solar Image Compression
Department of Computer Science and Electrical Engineering, West Virginia University, WV, USA(计算机科学与电气工程系,维珍尼亚大学);Department of Mechanical & Aerospace Engineering, West Virginia University, WV, USA(机械与航空航天工程系,维珍尼亚大学);NASA Goddard Space Flight Center, MD, USA(国家航空航天局戈达德空间飞行中心)
Volcanic Clouds Detection through QCNN and Geostationary Satellite Multispectral Imagery
Istituto Nazionale di Geofisica e Vulcanologia (INGV)(国家地球物理与火山学研究所);Thales Alenia Space Italia(泰雷兹阿莱尼亚空间意大利公司);Euro-Mediterranean Center on Climate Change(欧亚地中海气候变化中心);Università di Catania(卡塔尼亚大学);INFN(国家核物理研究所)
Analysis of Plant Nutrient Deficiencies Using Multi-Spectral Imaging and Optimized Segmentation Model
organization= Temasek Life Sciences Laboratory , country= Singapore;Disruptive \& Sustainable Technologies for Agricultural Precision, Singapore-MIT Alliance for Research;organization= Department of Biological Sciences, National University of Singapore , country= Singapore;organization= Department of Physics, University of Gothenburg , country= Sweden;organization= Science for Life Laboratory, Department of Physics, University of Gothenburg , country= Sweden