Achieving extremely high-quality and truly immersive interactive Virtual Reality (VR) is expected to require a wireless link to the cloud, providing multi-gigabit throughput and extremely low latency. A prime candidate for fulfilling these requirements is millimeter-wave (mmWave) communications, operating in the 30 to 300 GHz bands, rather than the traditional sub-6 GHz. Evaluations with first-generation mmWave Wi-Fi hardware, based on the IEEE 802.11ad standard, have so far largely remained limited to lower-layer metrics. In this work, we present the first experimental analysis of the capabilities of mmWave for streaming VR content, using a novel testbed capable of repeatably creating blockage through mobility. Using this testbed, we show that (a) motion may briefly interrupt transmission, (b) a broken line of sight may degrade throughput unpredictably, and (c) TCP-based streaming frameworks need careful tuning to behave well over mmWave.
Rate Splitting Multiple Access (RSMA) has emerged as an effective interference management scheme for applications that require high data rates. Although RSMA has shown advantages in rate enhancement and spectral efficiency, it has yet not to be ready for latency-sensitive applications such as virtual reality streaming, which is an essential building block of future 6G networks. Unlike conventional High-Definition streaming applications, streaming virtual reality applications requires not only stringent latency requirements but also the computation capability of the transmitter to quickly respo…
Contemporary Virtual Reality (VR) setups commonly consist of a Head-Mounted Display (HMD) tethered to a content-generating server. "Cutting the wire" in such setups and going truly wireless will require a wireless network capable of delivering enormous amounts of video data at an extremely low latency. Higher frequencies, such as the millimeter-wave (mmWave) band, can support these requirements. Due to high attenuation and path loss in the mmWave frequencies, beamforming is essential. For VR setups, beamforming must adapt in real-time to the user's head rotations, but can rely on the HMD's bui…
Virtual Reality (VR) has shown great potential to revolutionize the market by providing users immersive experiences with freedom of movement. Compared to traditional video streaming, VR is with ultra high-definition and dynamically changes with users' head and eye movements, which poses significant challenges for the realization of such potential. In this paper, we provide a detailed and systematic survey of enabling technologies of virtual reality and its applications in Internet of Things (IoT). We identify major challenges of virtual reality on system design, view prediction, computation, s…
Virtual reality (VR) video provides an immersive 360 viewing experience to a user wearing a head-mounted display: as the user rotates his head, correspondingly different fields-of-view (FoV) of the 360 video are rendered for observation. Transmitting the entire 360 video in high quality over bandwidth-constrained networks from server to client for real-time playback is challenging. In this paper we propose a multi-stream switching framework for VR video streaming: the server pre-encodes a set of VR video streams covering different view ranges that account for server-client round trip time (RTT…
5G communication brings substantial improvements in the quality of service provided to various applications by achieving higher throughput and lower latency. However, interactive multimedia applications (e.g., ultra high definition video conferencing, 3D and multiview video streaming, crowd-sourced video streaming, cloud gaming, virtual and augmented reality) are becoming more ambitious with high volume and low latency video streams putting strict demands on the already congested networks. Mobile Edge Computing (MEC) is an emerging paradigm that extends cloud computing capabilities to the edge…
Virtual reality systems today cannot yet stream immersive, retina-quality virtual reality video over a network. One of the greatest challenges to this goal is the sheer data rates required to transmit retina-quality video frames at high resolutions and frame rates. Recent work has leveraged the decay of visual acuity in human perception in novel gaze-contingent video compression techniques. In this paper, we show that reducing the motion-to-photon latency of a system itself is a key method for improving the compression ratio of gaze-contingent compression. Our key finding is that a client and …
Delivering 360 degree video streaming for virtual and augmented reality presents many technical challenges especially in bandwidth starved wireless environments. Recently, a so-called two-tier approach has been proposed which delivers a basic-tier chunk and select enhancement-tier chunks to improve user experience while reducing network resources consumption. The video chunks are to be transmitted via unicast or multicast over an ultra-dense small cell infrastructure with enough bandwidth where small cells store video chunks in local caches. In this setup, user-cell association algorithms play…
The wireless communications landscape is anticipated to offer new service levels following the introduction of the millimeter-wave (mmWave) spectrum to consumer electronics. With their broad bandwidths and corresponding multi-Gbps data rates, these mmWaves are a perfect fit for data hungry applications, such as streaming video to extended reality devices. However, the latter are also bound by maximal latency constraints as low as 1 ms. Understanding where such minuscule time delays lurk requires a close-up study of individual layers in the network stack. Starting from the bottom up, the presen…
The need for cables with high-fidelity Virtual Reality (VR) headsets remains a stumbling block on the path towards interactive multi-user VR. Due to strict latency constraints, designing fully wireless headsets is challenging, with the few commercially available solutions being expensive. These solutions use proprietary millimeter wave (mmWave) communications technologies, as extremely high frequencies are needed to meet the throughput and latency requirements of VR applications. In this work, we investigate whether such a system could be built using specification-compliant IEEE 802.11ad hardw…
The millimeter-wave (mmWave) technology, recently standardized by IEEE 802.11ad, is emerging as an attractive alternative to the traditional 2.4/5GHz wireless systems, promising multi-Gigabit wireless throughput. However, the high attenuation and vulnerability to blockage of 60 GHz links have limited its applications (until recently) to short-range, line-of-sight, static scenarios. On the other hand, the question of whether it is feasible to build general-purpose WLANs out of mmWave radios in dynamic indoor environments with non-line-of-sight links remains largely unanswered. In this paper, th…
Next-generation wireless communication technologies will allow users to obtain unprecedented performance, paving the way to new and immersive applications. A prominent application requiring high data rates and low communication delay is Virtual Reality (VR), whose presence will become increasingly stronger in the years to come. To the best of our knowledge, we propose the first traffic model for VR applications based on traffic traces acquired from a commercial VR streaming software, allowing the community to further study and improve the technology to manage this type of traffic. This work im…
In this paper, a novel proactive computing and mmWave communication for ultra-reliable and low latency wireless virtual reality (VR is proposed. By leveraging information about users' poses, proactive computing and caching are used to pre-compute and store users' HD video frames to minimize the computing latency. Furthermore, multi-connectivity is exploited to ensure reliable mmWave links to deliver users' requested HD frames. The performance of the proposed approach is validated on a VR network serving an interactive gaming arcade, where dynamic and real-time rendering of HD video frames is n…
Cloud virtual reality (VR) gaming traffic characteristics such as frame size, inter-arrival time, and latency need to be carefully studied as a first step toward scalable VR cloud service provisioning. To this end, in this paper we analyze the behavior of VR gaming traffic and Quality of Service (QoS) when VR rendering is conducted remotely in the cloud. We first build a VR testbed utilizing a cloud server, a commercial VR headset, and an off-the-shelf WiFi router. Using this testbed, we collect and process cloud VR gaming traffic data from different games under a number of network conditions …
eXtended reality (XR) technologies such as virtual reality and 360° stereoscopic streaming enable the concept of the Metaverse, an immersive virtual space for collaboration and interaction. To ensure high fidelity display of immersive media, the bandwidth, latency and network traffic patterns will need to be considered to ensure a user's Quality of Experience (QoE). In this article, examples and calculations are explored to demonstrate the requirements of the abovementioned parameters. Additionally, future methods such as network-awareness using reinforcement learning (RL) and XR content aware…
The next generation of multimedia applications will require the telecommunication networks to support a higher bitrate than today, in order to deliver virtual reality and ultra-high quality video content to the users. Most of the video content will be accessed from mobile devices, prompting the provision of very high data rates by next generation (5G) cellular networks. A possible enabler in this regard is communication at mmWave frequencies, given the vast amount of available spectrum that can be allocated to mobile users; however, the harsh propagation environment at such high frequencies ma…
Immersive media streaming, especially virtual reality (VR)/360-degree video streaming which is very bandwidth demanding, has become more and more popular due to the rapid growth of the multimedia and networking deployments. To better explore the usage of resource and achieve better quality of experience (QoE) perceived by users, this paper develops an application-layer scheme to jointly exploit the available bandwidth from the LTE and Wi-Fi networks in 360-degree video streaming. This newly proposed scheme and the corresponding solution algorithms utilize the saliency of video, prediction of u…
Internet video traffic has been been rapidly increasing and is further expected to increase with the emerging 5G applications such as higher definition videos, IoT and augmented/virtual reality applications. As end-users consume video in massive amounts and in an increasing number of ways, the content distribution network (CDN) should be efficiently managed to improve the system efficiency. The streaming service can include multiple caching tiers, at the distributed servers and the edge routers, and efficient content management at these locations affect the quality of experience (QoE) of the e…
In this paper, we consider the multi-user scheduling problem in millimeter wave (mmWave) video streaming networks, which comprise a streaming server and several users, each requesting a video stream with a different resolution. The main objective is to optimize the long-term average quality of experience (QoE) for all users. We tackle this problem by considering the physical layer characteristics of the mmWave network, including the beam alignment overhead due to pencil-beams. To develop an efficient scheduling policy, we leverage the contextual multi-armed bandit (MAB) models to propose a bea…
Singapore University of Technology and Design（新加坡科技设计大学）；Beihang University（北京航空航天大学）
This paper investigates the problem of providing ultra-reliable and energy-efficient virtual reality (VR) experiences for wireless mobile users. To ensure reliable ultra-high-definition (UHD) video frame delivery to mobile users and enhance their immersive visual experiences, a coordinated multipoint (CoMP) transmission technique and millimeter wave (mmWave) communications are exploited. Owing to user movement and time-varying wireless channels, the wireless VR experience enhancement problem is formulated as a sequence-dependent and mixed-integer problem with a goal of maximizing users' feelin…
Next-generation wireless networks must enable emerging technologies such as augmented reality and connected autonomous vehicles via wide range of wireless services that span enhanced mobile broadband (eMBB), as well as ultra-reliable low-latency communication (URLLC). Existing wireless systems that solely rely on the scarce sub-6 GHz, microwave ($μ$W) frequency bands will be unable to meet such stringent and mixed service requirements for future wireless services due to spectrum scarcity. Meanwhile, operating at high-frequency millimeter wave (mmWave) bands is seen as an attractive solution, p…
