Informative Data Selection with Uncertainty for Multi-modal Object Detection
Beihang University(北京航空航天大学);Tsinghua University(清华大学);China University of Mining & Technology, Beijing(中国矿业大学(北京));Beijing University of Chemical Technology(北京化工大学)
Xinyu Zhang, Zhiwei Li, Zhenhong Zou, Xin Gao, Yijin Xiong, Dafeng Jin, Jun Li, Huaping Liu
Noise has always been nonnegligible trouble in object detection by creating confusion in model reasoning, thereby reducing the informativeness of the data. It can lead to inaccurate recognition due to the shift in the observed pattern, that requires a robust generalization of the models. To implement a general vision model, we need to develop deep learning models that can adaptively select valid information from multi-modal data. This is mainly based on two reasons. Multi-modal learning can break through the inherent defects of single-modal data, and adaptive information selection can reduce chaos in multi-modal data. To tackle this problem, we propose a universal uncertainty-aware multi-modal fusion model. It adopts a multi-pipeline loosely coupled architecture to combine the features and results from point clouds and images. To quantify the correlation in multi-modal information, we model the uncertainty, as the inverse of data information, in different modalities and embed it in the bounding box generation. In this way, our model reduces the randomness in fusion and generates reliable output. Moreover, we conducted a completed investigation on the KITTI 2D object detection dataset and its derived dirty data. Our fusion model is proven to resist severe noise interference like Gaussian, motion blur, and frost, with only slight degradation. The experiment results demonstrate the benefits of our adaptive fusion. Our analysis on the robustness of multi-modal fusion will provide further insights for future research.
Multi-modal Uncertainty Robust Tree Cover Segmentation For High-Resolution Remote Sensing Images
School of Information and Electronics, Beijing Institute of Technology(信息与电子学院,北京理工大学);National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing(空间智能信息处理国家重点实验室);Department of Electrical and Computer Engineering, University of Delaware(电气与计算机工程系,德雷塞尔大学);Swiss Federal Institute for Forest, Snow, and Landscape Research WSL(瑞士森林、雪和景观研究联邦 institute WSL)
MMLF: Multi-modal Multi-class Late Fusion for Object Detection with Uncertainty Estimation
Glasgow College, University of Electronic Science and Technology of China(电子科技大学格拉斯哥学院);School of Automation Engineering, University of Electronic Science and Technology of China(电子科技大学自动化工程学院)
Universal Weighting Metric Learning for Cross-Modal Matching
Center for Future Media(未来媒体中心);School of Computer Science and Engineering(计算机科学与工程学院);University of Electronic Science and Technology of China(电子科技大学)
TMF-RSE: Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty for Lung Severity Scoring
Institute of Applied Sciences and Intelligent Systems (ISASI), CNR(应用科学与智能系统研究所(ISASI),CNR);Department of Computer Science and Artificial Intelligence, University of the Basque Country (UPV/EHU)(巴斯克国家大学计算机科学与人工智能系);University of Salento(萨莱nton大学)
UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification
Information Materials and Intelligent Sensing Laboratory of Anhui Province, School of Artificial Intelligence, Anhui University(安徽省信息材料与智能感知实验室,人工智能学院,安徽大学);Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University(安徽省多模态认知计算重点实验室,计算机科学与技术学院,安徽大学)
ELMAR: Enhancing LiDAR Detection with 4D Radar Motion Awareness and Cross-modal Uncertainty
Infineon Technologies AG and the Technical University of Munich(英飞凌科技有限公司和技术大学慕尼黑);China University of Geosciences(中国地质大学);Technical University of Munich(技术大学慕尼黑)
Shanghai Jiao Tong University(上海交通大学);University of Southern California(南加州大学);School of Mechanical Engineering, Nanjing University of Science and Technology(南京理工大学机械工程学院)
Multi-modal Evidential Fusion Network for Trustworthy PET/CT Tumor Segmentation
Soochow University(苏州大学);the University of Hong Kong(香港大学);Southern University of Science and Technology(南方科技大学);Changshu Institute of Technology(常熟理工学院);the First Affiliated Hospital of Soochow University(苏州大学附属第一医院)
Aligning First, Then Fusing: A Novel Weakly Supervised Multimodal Violence Detection Method
School of Software Engineering(软件工程学院);Shenzhen Institute of Advanced Technology(深圳先进技术研究院);Xi’an Jiaotong University(西安交通大学);Chinese Academy of Sciences(中国科学院)
HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval
School of Software Shandong University Jinan China(软件学院 山东大学 济南 中国);School of Data Science City University of Hong Kong Hong Kong China(数据科学学院 香港城市大学 香港 中国);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));City University of Hong Kong(香港城市大学)