演示:基于设备端意图感知语义分解的实时生成多播
Demo: Real-time Generative Multicasting with On-Device Intent-aware Semantic Decomposition
- University of Surrey(萨里大学)
机构由 AI 辅助整理,请以论文原文为准。
AI总结:
该研究提出基于设备端意图感知语义分解的实时生成多播系统,通过DNN分割与生成模型实现资源优化,在Google Coral Edge TPU上完成演示,推动设备端生成语义通信发展。
AI中文摘要:
我们展示了一项生成多播的演示,该演示采用设备端意图感知语义分解。在发射端,基于DNN的分割从源视频中提取语义图,根据多用户接收端意图将其分解为多个子信号类别。发射端通过共享无线/网络资源向所有用户广播语义图,仅使用正交资源传输面向每个用户的子信号类别。用户通过将接收到的目标类别与生成模型从语义图本地合成的非目标类别相结合,部分重建并部分合成信号。我们推导了采用生成模型进行重建/合成的率失真/感知曲线,以自适应设置语义图和目标类别的压缩率。生成多播可大幅降低现有/新兴多媒体多播应用所需的无线/网络资源。该系统在具有4 TOPS(int8)的Google Coral Edge TPU上实现实时运行,这是生成多播的首次演示,代表了设备端生成语义通信(SemCom)的重大进展。
英文摘要:
We present a demonstration for generative multicasting with on-device, intent-aware semantic decomposition. At the transmitter, DNN-based segmentation extracts a semantic map from the source video, decomposing it into multiple sub-signal classes based on multi-user receiver intents. The transmitter broadcasts the semantic map to all users over shared wireless/network resources, thereby utilizing orthogonal resources only to transmit the sub-signal classes intended for each user. Users partially reconstruct and partially synthesize the signal by combining the received intended classes with non-intended classes locally synthesized by a generative model from the semantic map. We derive the rate-distortion/perception curves for reconstruction/synthesis with the generative model, to adaptively set compression rates for the semantic map and intended classes. Generative multicasting significantly reduces the wireless/network resources required for existing/emerging multimedia multicasting applications. The system is real-time on a Google Coral Edge TPU with 4 TOPS (int8). This is the first demonstration of generative multicasting representing a substantial advancement in on-device generative SemCom.