arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.00964eess.SP
收藏

基于多模态融合的多任务语义通信系统

Multi-Modal Fusion-Based Multi-Task Semantic Communication System

School of Electronics, Peking University(北京大学电子学院);Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology(香港科技大学电子与计算机工程系)

Zengle Zhu, Rongqing Zhang, Xiang Cheng, Liuqing Yang

AI总结:

本文提出基于多模态融合的多任务语义通信系统,利用BERT融合模块提升多模态信息处理效率,实验表明其在性能和通信开销上优于现有模型。

摘要:

近年来,深度学习技术显著推动了语义通信系统的发展,极大提高了信息传输效率。然而,传统语义通信模型仍面临挑战,特别是其单任务和单模态的局限性。许多模型专门针对特定任务设计,可能在多任务通信系统中受限。此外,这些模型往往忽视多模态任务中不同数据模态之间的相关性,导致对复杂信息的理解不完整,增加通信开销并降低性能。为解决这些问题,我们提出了一种基于多模态融合的多任务语义通信(MFMSC)框架。与传统语义通信方法不同,MFMSC能够有效处理多种模态下的各种任务。此外,我们设计了一个基于双向编码器表示变换器(BERT)的融合模块,用于多模态语义信息融合。通过利用BERT强大的语义理解能力和自注意力机制,我们实现了不同模态语义信息的有效融合。我们比较了我们的模型与多个基准模型。仿真结果表明,MFMSC在性能和通信开销方面均优于这些模型。

英文摘要:

In recent years, there has been significant progress in semantic communication systems empowered by deep learning techniques. It has greatly improved the efficiency of information transmission. Nevertheless, traditional semantic communication models still face challenges, particularly due to their single-task and single-modal orientation. Many of these models are designed for specific tasks, which may result in limitations when applied to multi-task communication systems. Moreover, these models often overlook the correlations among different modal data in multi-modal tasks. It leads to an incomplete understanding of complex information, causing increased communication overhead and diminished performance. To address these problems, we propose a multi-modal fusion-based multi-task semantic communication (MFMSC) framework. In contrast to traditional semantic communication approaches, MFMSC can effectively handle various tasks across multiple modalities. Furthermore, we design a fusion module based on Bidirectional Encoder Representations from Transformers (BERT) for multi-modal semantic information fusion. By leveraging the powerful semantic understanding capabilities and self-attention mechanism of BERT, we achieve effective fusion of semantic information from different modalities. We compare our model with multiple benchmarks. Simulation results show that MFMSC outperforms these models in terms of both performance and communication overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑