The objective of multimodal information fusion is to mathematically analyze information carried in different sources and create a new representation which will be more effectively utilized in pattern recognition and other multimedia information processing tasks. In this paper, we introduce a new method for multimodal information fusion and representation based on the Labeled Multiple Canonical Correlation Analysis (LMCCA). By incorporating class label information of the training samples,the proposed LMCCA ensures that the fused features carry discriminative characteristics of the multimodal information representations, and are capable of providing superior recognition performance. We implement a prototype of LMCCA to demonstrate its effectiveness on handwritten digit recognition,face recognition and object recognition utilizing multiple features,bimodal human emotion recognition involving information from both audio and visual domains. The generic nature of LMCCA allows it to take as input features extracted by any means,including those by deep learning (DL) methods. Experimental results show that the proposed method enhanced the performance of both statistical machine learning (SML) methods, and methods based on DL.
Generalizable Person Re-Identification via Viewpoint Alignment and Fusion
Northwestern Polytechnical University(西北工业大学);Ningbo Institute, Northwestern Polytechnical University(西北工业大学宁波研究院);Nanyang Technological University(南洋理工大学);The University of Adelaide(阿德莱德大学)
Dr. B. C. Roy Engineering College(B.C.罗伊工程学院);National Institute of Technology Rourkela(鲁尔克拉国家理工学院);Indian Institute of Technology Kanpur(坎普尔印度理工学院);Jadavpur University(贾达普大学)
A Survey: Deep Learning for Hyperspectral Image Classification with Few Labeled Samples
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);SZU Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳市人工智能与机器人研究院深圳大学分部);Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程系)
College of Safety Science and Engineering, Civil Aviation University of China(中国民航大学安全科学与工程学院);College of Computer Science and Technology, Civil Aviation University of China(中国民航大学计算机科学与技术学院);North China Electric Power University(华北电力大学)
AttentionLut: Attention Fusion-based Canonical Polyadic LUT for Real-time Image Enhancement
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University(上海交通大学图像通信与网络工程研究所);Transsion(传音控股);John Hopcroft Center, Shanghai Jiao Tong University(上海交通大学约翰·霍普克罗夫特中心)
Pseudo Label-Guided Data Fusion and Output Consistency for Semi-Supervised Medical Image Segmentation
college of physics and information engineering, University of Fuzhou University(福州大学物理与信息工程学院);Faculty of Applied Science, University of Macao Polytechnic University(澳门理工大学应用科学学院);University of Toronto University(多伦多大学)
Meningioma Analysis and Diagnosis using Limited Labeled Samples
Shaanxi University of Science and Technology(陕西科技大学);Department of Neurosurgery, The First Affiliated Hospital of Xi'an Jiaotong University(西安交通大学第一附属医院神经外科);Department of Neurosurgery, The First Affiliated Hospital of Fujian Medical University(福建医科大学第一附属医院神经外科);Department of Radiotherapy, University of Groningen, University Medical Center Groningen(格罗宁根大学放射治疗科,格罗宁根大学医学中心);Department of Nephrology, Chenggong Hospital Affiliated to Xiamen University(厦门大学附属晋江医院肾内科)
FFM-CP: Cross-Backbone Fusion of Vision-Language Foundation Models for Few-Shot Computational Pathology
Giessen University(吉森大学);University Medicine Gottingen(哥廷根大学医学中心);Technical University of Denmark(丹麦技术大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Max Planck Institute for Biology of Ageing(马克斯·普朗克衰老生物学研究所);National Taiwan University(国立台湾大学);VNU University of Medicine and Pharmacy(越南国立大学医学与药学大学);University of Medicine and Pharmacy, Hue University(顺化大学医学与药学大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);University of Stuttgart(斯图加特大学)
Exploring Modality Guidance to Enhance VFM-based Feature Fusion for UDA in 3D Semantic Segmentation
Institute of Visual Computing, Graz University of Technology(视觉计算研究所,格拉茨技术大学);JKU LINZ(JKU林茨);TU Wien(维也纳技术大学);IBM Research(IBM研究院);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)
An Enhanced Classification Method Based on Adaptive Multi-Scale Fusion for Long-tailed Multispectral Point Clouds
School of Electronics and Information Engineering(电子信息工程学院);Harbin Institute of Technology(哈尔滨工业大学);the Department of Telecommunications and Information Processing(电信与信息处理系)
Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancement
College of Computer Science, Sichuan University(四川大学计算机科学学院);Engineering Research Center of Machine Learning(机器学习工程研究中心);China Telecom Institute of AI(中国电信人工智能研究院)
Multimodal Latent Fusion of ECG Leads for Early Assessment of Pulmonary Hypertension
School of Computer Science, University of Sheffield(谢菲尔德大学计算机科学学院);Centre for Machine Intelligence, University of Sheffield(谢菲尔德大学智能中心);Department of Electrical & Computer Science Engineering, IITRAM Ahmedabad(印度阿赫迈德亚布德IITRAM电子与计算机科学工程系);Digital Environment Research Institute, Queen Mary University of London(伦敦玛丽女王大学数字环境研究院);Nepal Applied Mathematics and Informatics Institute for research (NAAMII), Nepal(尼泊尔应用数学与信息技术研究所);Department of Computing, Imperial College London(伦敦帝国学院计算机系);School of Medicine and Population Health, University of Sheffield(谢菲尔德大学医学与人口健康学院);Department of Clinical Radiology, Sheffield Teaching Hospitals(谢菲尔德教学医院放射科);National Institute for Health and Care Research (NIHR), Sheffield Biomedical Research Centre(英国国家健康与护理研究所(NIHR)谢菲尔德生物医学研究中心)
Robust Incomplete-Modality Alignment for Ophthalmic Disease Grading and Diagnosis via Labeled Optimal Transport
University of Exeter(埃克塞特大学);University of Liverpool(利物浦大学);Chinese Academy of Sciences(中国科学院);The University of Hong Kong(香港大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Dalian University of Technology(大连理工大学);A*STAR
Class-Aware Mask-Guided Feature Refinement for Scene Text Recognition
School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);Huawei Inc.(华为公司);School of Software Engineering, Huazhong University of Science and Technology(华中科技大学软件学院)