Task-oriented image semantic communication is a new communication paradigm, which aims to transmit semantics for artificial intelligent (AI) tasks while ignoring the reconstruction quality of the images. However, in some applications, such as autonomous driving, both image reconstruction quality and the performance of the followed AI tasks must be simultaneously considered. To tackle this challenge, this paper proposes a task-oriented semantic communication scheme with semantic reconstruction (TOSC-SR). Its main goal is to simultaneously minimize pixel-level and task-relevant semantic-level distortion during communications under a certain rate, which formulates a new rate-distortion optimization problem. To successfully measure the loss at the semantic level, a new form of semantic distortion measured by the mutual information between the semantic-reconstructed images and the task labels is proposed. Then, we derive an analytical solution for the formulated problem, where the self-consistent equations of the problem are obtained to determine the optimal mapping of the source and the semantic-reconstructed images. To implement TOSC-SR, we further obtain an extended form of rate-distortion form based on the variational approximation of mutual information, which is applicable to multiple AI tasks. Experimental results show that the proposed approach outperforms the traditional JPEG, JPEG2000, BPG, VVC-based image communication systems and deep learning based benchmarks in terms of image reconstruction quality, AI task performance, and multi-task generalization ability.
Using the Semantic Information G Measure to Explain and Extend Rate-Distortion Functions and Maximum Entropy Distributions
In the rate-distortion function and the Maximum Entropy (ME) method, Minimum Mutual In-formation (MMI) distributions and ME distributions are expressed by Bayes-like formulas, in-cluding Negative Exponential Functions (NEFs) and partition functions. Why do these non-probability functions exist in Bayes-like formulas? On the other hand, the rate-distortion function has three disadvantages: (1) the distortion function is subjectively defined; (2) the defi-nition of the distortion function between instances and labels is often difficult; (3) it cannot be used for data compression according to the…
Fundamental Limitation of Semantic Communications: Neural Estimation for Rate-Distortion
Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Future Network of Intelligence Institute（未来智联网络研究院）；University of Hong Kong（香港大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；State Key Laboratory of Networking and Switching Technology（网络与交换技术国家重点实验室）
Distortion Resilience for Goal-Oriented Semantic Communication
Department of Information Convergence Engineering, Pusan National University（信息融合工程系，釜山国立大学）；Wireless Communications Research Group, Faculty of Electrical and Electronics Engineering, Ton Duc Thang University（无线通信研究组，电子工程学院， Ton Duc Thang 大学）；College of Information Science and Electronic Engineering, Zhejiang University（信息科学与电子工程学院，浙江大学）；School of Computer Science and Statistics, Trinity College Dublin, The University of Dublin（计算机科学与统计学系，三一学院，都柏林大学）
Distributed Electromagnetic Neural Networks for Task-Oriented Semantic Communications
University of Electronic Science and Technology of China（电子科技大学）；The University of British Columbia（不列颠哥伦比亚大学）；University of Oulu（奥卢大学）；Khalifa University（哈利法大学）
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Imperial College London（帝国理工学院）；Tsinghua University（清华大学）；Amazon（亚马逊）；Virginia Tech（弗吉尼亚理工学院）；Zhejiang Lab（之江实验室）；Zhejiang University（浙江大学）；The Ohio State University（俄亥俄州立大学）；University College London（伦敦大学学院）；Yonsei University（延世大学）
