Generative models are typically evaluated by direct inspection of their generated samples, e.g., by visual inspection in the case of images. Further evaluation metrics like the Fréchet inception distance or maximum mean discrepancy are intricate to interpret and lack physical motivation. These observations make evaluating generative models in the wireless PHY layer non-trivial. This work establishes a framework consisting of evaluation metrics and methods for generative models applied to the wireless PHY layer. The proposed metrics and methods are motivated by wireless applications, facilitating interpretation and understandability for the wireless community. In particular, we propose a spectral efficiency analysis for validating the generated channel norms and a codebook fingerprinting method to validate the generated channel directions. Moreover, we propose an application cross-check to evaluate the generative model's samples for training machine learning-based models in relevant downstream tasks. Our analysis is based on real-world measurement data and includes the Gaussian mixture model, variational autoencoder, diffusion model, and generative adversarial network as generative models. Our results under a fair comparison in terms of model architecture indicate that solely relying on metrics like the maximum mean discrepancy produces insufficient evaluation outcomes. In contrast, the proposed metrics and methods exhibit consistent and explainable behavior.
Generative Diffusion Models for High Dimensional Channel Estimation
National Mobile Communications Research Laboratory, Southeast University（东南大学移动通信国家重点实验室）；Purple Mountain Laboratories（紫金山实验室）；National Key Laboratory of Advanced Communication Networks（高级通信网络全国重点实验室）；Academy of Network & Communications of CETC（中国电子科技集团公司网络通信研究院）
JSCGC: Joint Source-Channel-Generation Coding for Wireless Generative Communications
Cooperative Medianet Innovation Center, the School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（联合中位网创新中心，信息科学与电子工程学院，上海交通大学）
One-Step Generative Channel Estimation via Average Velocity Field
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息与电子工程学院）；Guangdong Tobacco Maoming Co., Ltd.（广东烟草茂名有限责任公司）；KU 6G Research Center, Khalifa University of Science and Technology（哈利法科技大学 KU 6G 研究中心）
Communicate Less, Synthesize the Rest: Latency-aware Intent-based Generative Semantic Multicasting with Diffusion Models
GIC & 6GIC, Institute for Communication Systems (ICS), University of Surrey（5GIC与6GIC，通信系统研究所（ICS），萨里大学）；School of Information and Electronics, Beijing Institute of Technology（信息电子学院，北京理工大学）；Centre for Wireless Communications, University of Oulu（无线通信中心，奥卢大学）
Cache-enabled Generative Joint Source-Channel Coding for Evolving Semantic Communications
Zhejiang University（浙江大学）；Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）；The University of Hong Kong（香港大学）；Imperial College London（帝国理工学院）
Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communications
Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）；Tsinghua University（清华大学）；Hong Kong University of Science and Technology（香港科技大学）
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；National Mobile Communications Research Laboratory, Southeast University（国家移动通信研究中心，东南大学）
