arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.18005cs.ITcs.LGmath.IT
收藏

生成式语义通信用于联合图像传输与分割

Generative Semantic Communication for Joint Image Transmission and Segmentation

The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));Nanyang Technological University(南洋理工大学);Shenzhen University(深圳大学);Sungkyunkwan University(成均馆大学)

Weiwen Yuan, Jinke Ren, Chongjie Wang, Ruichen Zhang, Jun Wei, Dong In Kim, Shuguang Cui

AI总结:

提出一种基于语义知识库和生成式AI(Swin-Transformer、扩散模型)的多任务生成式语义通信系统,联合实现图像传输与分割,在峰值信噪比和分割精度上优于单任务系统。

摘要:

语义通信已成为一种有前景的技术,用于提升通信效率。然而,现有研究大多强调单任务重建,忽视了模型在多任务系统中的适应性和泛化能力。本文提出了一种新颖的生成式语义通信系统,支持图像重建和分割两种任务。我们的方法在发射机和接收机均基于语义知识库(KBs),每个语义知识库包含一个源知识库和一个任务知识库。发射端的源知识库利用分层Swin-Transformer(一种生成式AI方案)从输入图像中提取多级特征。同时,接收端的对应源知识库利用分层残差块生成任务特定知识。此外,任务知识库采用语义相似度模型将不同的任务需求映射为预定义的任务指令,从而促进源知识库的特征选择。我们还开发了基于统一残差块的联合源信道编码(JSCC)编码器和两个任务特定的JSCC解码器,以实现这两种图像任务。特别地,采用生成式扩散模型构建用于图像重建任务的JSCC解码器。实验结果表明,我们的多任务生成式语义通信系统在峰值信噪比和分割精度方面优于先前的单任务通信系统。

英文摘要:

Semantic communication has emerged as a promising technology for enhancing communication efficiency. However, most existing research emphasizes single-task reconstruction, neglecting model adaptability and generalization across multi-task systems. In this paper, we propose a novel generative semantic communication system that supports both image reconstruction and segmentation tasks. Our approach builds upon semantic knowledge bases (KBs) at both the transmitter and receiver, with each semantic KB comprising a source KB and a task KB. The source KB at the transmitter leverages a hierarchical Swin-Transformer, a generative AI scheme, to extract multi-level features from the input image. Concurrently, the counterpart source KB at the receiver utilizes hierarchical residual blocks to generate task-specific knowledge. Furthermore, the task KBs adopt a semantic similarity model to map different task requirements into pre-defined task instructions, thereby facilitating the feature selection of the source KBs. Additionally, we develop a unified residual block-based joint source and channel (JSCC) encoder and two task-specific JSCC decoders to achieve the two image tasks. In particular, a generative diffusion model is adopted to construct the JSCC decoder for the image reconstruction task. Experimental results show that our multi-task generative semantic communication system outperforms previous single-task communication systems in terms of peak signal-to-noise ratio and segmentation accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑