本文提出了一种基于矢量量化变分自编码器(Vector-Quantized Variational Autoencoder,VQVAE)架构的新型多模态数据融合方法。该方法简洁有效,在配对的MNIST-SVHN数据和WiFi频谱图数据上均实现了出色的重建性能。此外,本文将多模态VQVAE模型扩展至5G通信场景,构建了端到端信道状态信息(Channel State Information,CSI)反馈系统,用于压缩基站(eNodeB)与用户设备(User Equipment,UE)之间传输的数据,且不会造成显著的性能损失。该模型能为各类输入数据(CSI、频谱图、自然图像等)学习到具有判别性的压缩特征空间,适用于计算资源有限的应用场景。
英文摘要:
This paper presents a novel approach for multimodal data fusion based on the Vector-Quantized Variational Autoencoder (VQVAE) architecture. The proposed method is simple yet effective in achieving excellent reconstruction performance on paired MNIST-SVHN data and WiFi spectrogram data. Additionally, the multimodal VQVAE model is extended to the 5G communication scenario, where an end-to-end Channel State Information (CSI) feedback system is implemented to compress data transmitted between the base-station (eNodeB) and User Equipment (UE), without significant loss of performance. The proposed model learns a discriminative compressed feature space for various types of input data (CSI, spectrograms, natural images, etc), making it a suitable solution for applications with limited computational resources.
Environment-Aware Channel Inference via Cross-Modal Flow: From Multimodal Sensing to Wireless Channels
School of Computing Science, University of Glasgow(格拉斯哥大学计算机科学学院);School of Electronics and Computer Science, University of Southampton(南安普顿大学电子与计算机科学学院)
Orchestrating Multimodal DNN Workloads in Wireless Neural Processing
Department of Electronic and Electrical Engineering, University College London(电子与电气工程系,伦敦大学学院);Department of Electronic Engineering, Kyung Hee University(电子工程系,庆熙大学);School of Electrical and Electronic Engineering, the University of Sheffield(电气与电子工程学院,谢菲尔德大学);Department of Electronics and Information Convergence Engineering, Kyung Hee University(电子与信息融合工程系,庆熙大学)
Large AI Model Empowered Multimodal Semantic Communications
Hunan Normal University(湖南师范大学);Hunan University of Technology and Business(湖南工商大学);Brunel University(布鲁内尔大学);Nanjing University(南京大学);Southeast University(东南大学)
A Transformer-based Multimodal Fusion Model for Efficient Crowd Counting Using Visual and Wireless Signals
School of Electrical and Electronic Engineering, University College Dublin(都柏林大学电子与电气工程学院);College of Electrical Engineering and Automation, Shandong University of Science and Technology(山东科技大学电气工程与自动化学院)
Research Institute for Digital Future, Khalifa University(数字未来研究院,哈利法大学);College of Information Science and Electronic Engineering, Zhejiang University(信息科学与电子工程学院,浙江大学);Université de Lorraine, CNRS, CRAN(洛林大学,CNRS,CRAN)
Multimodal LLM Integrated Semantic Communications for 6G Immersive Experiences
School of Electronic and Information Engineering, Beijing Jiaotong University(电子信息工程学院,北京交通大学);School of Electrical and Electronic Engineering, Imperial College London(电子电气工程学院,帝国理工学院伦敦分校)
Map as a Prompt: Learning Multi-Modal Spatial-Signal Foundation Models for Cross-scenario Wireless Localization
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Pengcheng Laboratory(鹏城实验室);Shanghai Academy of AI for Science(上海人工智能科学研究院);Artificial Intelligence Innovation and Incubation Institute, Fudan University(复旦大学人工智能创新与孵化院)
A Novel Spatiotemporal Correlation Anomaly Detection Method Based on Time-Frequency-Domain Feature Fusion and a Dynamic Graph Neural Network in Wireless Sensor Network
Agentic TinyML for Intent-aware Handover in 6G Wireless Networks
Center for Ubiquitous Computing(无处不在计算中心);University of Oulu(奥卢大学);Department of Communication Systems(通信系统系);EURECOM;Department of Computer Science(计算机科学系);University of Helsinki(赫尔辛基大学);RISE Research Institutes of Sweden(瑞典研究机构);Department of Computer Science, Electrical and Space Engineering(计算机科学、电气与空间工程系);Luleå University of Technology(卢勒奥技术大学)
CSI-tuples-based 3D Channel Fingerprints Construction Assisted by MultiModal Learning
National Mobile Communications Research Laboratory, Southeast University(东南大学国家移动通信研究中心);Purple Mountain Laboratories(紫金山实验室);Huawei Technologies Co., Ltd.(华为技术有限公司)
Multi-Modal Sensing and Fusion in mmWave Beamforming for Connected Vehicles: A Transformer Based Framework
Department of Electrical and Computer Engineering, University of Massachusetts Dartmouth(电子与计算机工程系,马萨诸塞大学达特茅斯分校);Department of Graduate Computer Science and Engineering, Katz School of Science and Health, Yeshiva University(研究生计算机科学与工程系,耶鲁大学科学与健康学院)