Generative Semantic Communication for Joint Image Transmission and Segmentation
The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));Nanyang Technological University(南洋理工大学);Shenzhen University(深圳大学);Sungkyunkwan University(成均馆大学)
Weiwen Yuan, Jinke Ren, Chongjie Wang, Ruichen Zhang, Jun Wei, Dong In Kim, Shuguang Cui
备注:
This paper has been accepted by the 2025 IEEE International Conference on Communications Workshops and is scheduled for publication
Semantic communication has emerged as a promising technology for enhancing communication efficiency. However, most existing research emphasizes single-task reconstruction, neglecting model adaptability and generalization across multi-task systems. In this paper, we propose a novel generative semantic communication system that supports both image reconstruction and segmentation tasks. Our approach builds upon semantic knowledge bases (KBs) at both the transmitter and receiver, with each semantic KB comprising a source KB and a task KB. The source KB at the transmitter leverages a hierarchical Swin-Transformer, a generative AI scheme, to extract multi-level features from the input image. Concurrently, the counterpart source KB at the receiver utilizes hierarchical residual blocks to generate task-specific knowledge. Furthermore, the task KBs adopt a semantic similarity model to map different task requirements into pre-defined task instructions, thereby facilitating the feature selection of the source KBs. Additionally, we develop a unified residual block-based joint source and channel (JSCC) encoder and two task-specific JSCC decoders to achieve the two image tasks. In particular, a generative diffusion model is adopted to construct the JSCC decoder for the image reconstruction task. Experimental results show that our multi-task generative semantic communication system outperforms previous single-task communication systems in terms of peak signal-to-noise ratio and segmentation accuracy.
Comparative analysis of segmentation and generative models for fingerprint retrieval task
Birla Institute of Technology and Science Pilani – KK Birla Goa Campus(贝拉理工学院皮拉尼分校–KK·贝拉果阿校区);Indian Institute of Science Education and Research, Pune(印度科学教育与研究学院浦那分校);Vellore Institute of Technology, Bhopal(韦洛尔理工学院博帕尔分校)
Segmentation-guided Domain Adaptation and Data Harmonization of Multi-device Retinal Optical Coherence Tomography using Cycle-Consistent Generative Adversarial Networks
Simon Fraser University(西蒙弗雷泽大学);Wake Forest University School of Medicine(维克森林大学医学院);Wake Forest School of Medicine(维克森林大学医学院);University of Nottingham(诺丁汉大学);Tencent Jarvis Lab(腾讯Jarvis实验室);Memorial University of Newfoundland(纽芬兰纪念大学);University of British Columbia(不列颠哥伦比亚大学);The University of British Columbia(不列颠哥伦比亚大学);University College London(伦敦大学学院)