面向元宇宙的联邦多视图合成
Federated Multi-View Synthesizing for Metaverse
AI总结:
针对无线网络VR传输数据与计算密集导致服务质量难保障的问题,提出基于3D感知生成模型的联邦多视图合成框架,利用单视图图像传输和联邦学习降低通信延迟,实现高效元宇宙内容交付。
摘要:
元宇宙有望提供沉浸式娱乐、教育和商业应用。然而,无线网络上的虚拟现实(VR)传输是数据密集和计算密集型的,因此引入满足严格服务质量要求的新颖解决方案至关重要。借助边缘智能和深度学习的最新进展,我们开发了一种新颖的多视图合成框架,能够为元宇宙中的无线内容交付高效提供计算、存储和通信资源。我们提出了一种三维(3D)感知生成模型,该模型使用单视图图像集合。这些单视图图像被传输给一组具有重叠视野的用户,与传输切片或整个3D模型相比,避免了海量内容传输。随后,我们提出了一种联邦学习方法以保证高效的学习过程。通过用大潜在特征空间表征垂直和水平数据样本,可以提升训练性能,同时通过在联邦学习期间减少传输参数的数量,可以实现低延迟通信。我们还提出了一种联邦迁移学习框架,以实现对不同目标域的快速域适应。仿真结果证明了我们提出的联邦多视图合成框架在VR内容交付中的有效性。
英文摘要:
The metaverse is expected to provide immersive entertainment, education, and business applications. However, virtual reality (VR) transmission over wireless networks is data- and computation-intensive, making it critical to introduce novel solutions that meet stringent quality-of-service requirements. With recent advances in edge intelligence and deep learning, we have developed a novel multi-view synthesizing framework that can efficiently provide computation, storage, and communication resources for wireless content delivery in the metaverse. We propose a three-dimensional (3D)-aware generative model that uses collections of single-view images. These single-view images are transmitted to a group of users with overlapping fields of view, which avoids massive content transmission compared to transmitting tiles or whole 3D models. We then present a federated learning approach to guarantee an efficient learning process. The training performance can be improved by characterizing the vertical and horizontal data samples with a large latent feature space, while low-latency communication can be achieved with a reduced number of transmitted parameters during federated learning. We also propose a federated transfer learning framework to enable fast domain adaptation to different target domains. Simulation results have demonstrated the effectiveness of our proposed federated multi-view synthesizing framework for VR content delivery.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。