Supervised Learning based QoE Prediction of Video Streaming in Future Networks: A Tutorial with Comparative Study
Cardiff Metropolitan University（卡迪夫城市大学）；University of Western Australia（西澳大利亚大学）；Dublin City University（都柏林城市大学）；University College Dublin（都柏林大学学院）；University of Cagliari（卡利亚里大学）
Arslan Ahmad, Atif Bin Mansoor, Alcardo Alex Barakabitze, Andrew Hines, Luigi Atzori, Ray Walshe
发表信息：
IEEE Communications Magazine, vol. 59, no. 11, pp. 88-94, November 2021
The Quality of Experience (QoE) based service management remains key for successful provisioning of multimedia services in next-generation networks such as 5G/6G, which requires proper tools for quality monitoring, prediction and resource management where machine learning (ML) can play a crucial role. In this paper, we provide a tutorial on the development and deployment of the QoE measurement and prediction solutions for video streaming services based on supervised learning ML models. Firstly, we provide a detailed pipeline for developing and deploying supervised learning-based video streaming QoE prediction models which covers several stages including data collection, feature engineering, model optimization and training, testing and prediction and evaluation. Secondly, we discuss the deployment of the ML model for the QoE prediction/measurement in the next generation networks (5G/6G) using network enabling technologies such as Software-Defined Networking (SDN), Network Function Virtualization (NFV) and Mobile Edge Computing (MEC) by proposing reference architecture. Thirdly, we present a comparative study of the state-of-the-art supervised learning ML models for QoE prediction of video streaming applications based on multiple performance metrics.
We provide in this paper a tutorial and a comprehensive survey of QoE management solutions in current and future networks. We start with a high level description of QoE management for multimedia services, which integrates QoE modelling, monitoring, and optimization. This followed by a discussion of HTTP Adaptive Streaming (HAS) solutions as the dominant technique for streaming videos over the best-effort Internet. We then summarize the key elements in SDN/NFV along with an overview of ongoing research projects, standardization activities and use cases related to SDN, NFV, and other emerging ap…
The fundamental conflict between the enormous space of adaptive streaming videos and the limited capacity for subjective experiment casts significant challenges to objective Quality-of-Experience (QoE) prediction. Existing objective QoE models exhibit complex functional form, failing to generalize well in diverse streaming environments. In this study, we propose an objective QoE model namely knowledge-driven streaming quality index (KSQI) to integrate prior knowledge on the human visual system and human annotated data in a principled way. By analyzing the subjective characteristics towards str…
We take an analytical approach to study Quality of user Experience (QoE) for video streaming applications. First, we show that random linear network coding applied to blocks of video frames can significantly simplify the packet requests at the network layer and save resources by avoiding duplicate packet reception. Network coding allows us to model the receiver's buffer as a queue with Poisson arrivals and deterministic departures. We consider the probability of interruption in video playback as well as the number of initially buffered packets (initial waiting time) as the QoE metrics. We char…
The Quality of Experience (QoE) of streaming service is often degraded by frequent playback interruptions. To mitigate the interruptions, the media player prefetches streaming contents before starting playback, at a cost of delay. We study the QoE of streaming from the perspective of flow dynamics. First, a framework is developed for QoE when streaming users join the network randomly and leave after downloading completion. We compute the distribution of prefetching delay using partial differential equations (PDEs), and the probability generating function of playout buffer starvations using ord…
The popularity of the video services on the Internet has evolved various mechanisms that target the Quality of Experience (QoE) optimization of video traffic. The video quality has been enhanced through adapting the sending bitrates. However, rate adaptation alone is not sufficient for maintaining a good video QoE when congestion occurs. This paper presents a cross-layer architecture for video streaming that is QoE-aware. It combines adaptation capabilities of video applications and QoE-aware admission control to optimize the trade-off relationship between QoE and the number of admitted sessio…
University of New South Wales（新南威尔士大学）；BITS Pilani（皮拉尼比尔拉理工学院）
Social media, professional sports, and video games are driving rapid growth in live video streaming, on platforms such as Twitch and YouTube Live. Live streaming experience is very susceptible to short-time-scale network congestion since client playback buffers are often no more than a few seconds. Unfortunately, identifying such streams and measuring their QoE for network management is challenging, since content providers largely use the same delivery infrastructure for live and video-on-demand (VoD) streaming, and packet inspection techniques (including SNI/DNS query monitoring) cannot alway…
The knowledge of future throughput variations in mobile networks becomes more and more possible today thanks to the rich contextual information provided by mobile applications and services and smartphone sensors. It is even likely that such contextual information, which may include traffic, mobility and radio conditions will lead to a novel agile resource management not yet thought of. In this paper, we propose an framework (called NEWCAST) that anticipates the throughput variations to deliver video streaming content. We develop an optimization problem that realizes a fundamental trade-off amo…
HTTP adaptive streaming (HAS) is quickly becoming the dominant video delivery technique for adaptive streaming over the Internet. Still considered as its primary challenges are determining the optimal rate adaptation and improving both the quality of experience (QoE) and QoE-fairness. Most of the proposed approaches have relied on local information to find a result. However, employing techniques that provide a comprehensive and central view of the network resources can lead to more gains in performance. By leveraging software defined networking (SDN), this paper proposes an SDN-based framework…
Driven by the tremendous technological advancement of personal devices and the prevalence of wireless mobile network accesses, the world has witnessed an explosion in crowdsourced live streaming. Ensuring a better viewers quality of experience (QoE) is the key to maximize the audiences number and increase streaming providers' profits. This can be achieved by advocating a geo-distributed cloud infrastructure to allocate the multimedia resources as close as possible to viewers, in order to minimize the access delay and video stalls. Moreover, allocating the exact needed resources beforehand avoi…
Multimedia streaming to mobile devices is challenging for two reasons. First, the way content is delivered to a client must ensure that the user does not experience a long initial playback delay or a distorted playback in the middle of a streaming session. Second, multimedia streaming applications are among the most energy hungry applications in smartphones. The energy consumption mostly depends on the delivery techniques and on the power management techniques of wireless access technologies (Wi-Fi, 3G, and 4G). In order to provide insights on what kind of streaming techniques exist, how they …
Our purpose in this paper is to characterize buffer starvations for streaming services. The buffer is modeled as an M/M/1 queue, plus the consideration of bursty arrivals. When the buffer is empty, the service restarts after a certain amount of packets are \emph{prefetched}. With this goal, we propose two approaches to obtain the \emph{exact distribution} of the number of buffer starvations, one of which is based on \emph{Ballot theorem}, and the other uses recursive equations. The Ballot theorem approach gives an explicit result. We extend this approach to the scenario with a constant playbac…
We propose a robust scheme for streaming 360-degree immersive videos to maximize the quality of experience (QoE). Our streaming approach introduces a holistic analytical framework built upon the formal method of stochastic optimization. We propose a robust algorithm which provides a streaming rate such that the video quality degrades below that rate with very low probability even in presence of uncertain head movement, and bandwidth. It assumes the knowledge of the viewing probability of different portions (tiles) of a panoramic scene. Such probabilities can be easily derived from crowdsourced…
Immersive media streaming, especially virtual reality (VR)/360-degree video streaming which is very bandwidth demanding, has become more and more popular due to the rapid growth of the multimedia and networking deployments. To better explore the usage of resource and achieve better quality of experience (QoE) perceived by users, this paper develops an application-layer scheme to jointly exploit the available bandwidth from the LTE and Wi-Fi networks in 360-degree video streaming. This newly proposed scheme and the corresponding solution algorithms utilize the saliency of video, prediction of u…
Inferring the quality of network services is the vital basis of optimization for network operators. However, prevailing real-time video streaming applications adopt encryption for security, leaving it a problem to extract Quality of Service (QoS) indicators of real-time video. In this paper, we propose DaI, a traffic-based real-time video quality estimator. DaI can partially decrypt the encrypted real-time video data and applies machine learning methods to estimate key objective Quality of Experience (QoE) metrics of real-time video. According to the experimental results, DaI can estimate obje…
Unraveling quality of experience (QoE) of video streaming is very challenging in bandwidth shared wireless networks. It is unclear how QoE metrics such as starvation probability and buffering time interact with dynamics of streaming traffic load. In this paper, we collect view records from one of the largest streaming providers in China over two weeks and perform an in-depth measurement study on flow arrival and viewing time that shed light on the real traffic pattern. Our most important observation is that the viewing time of streaming users fits a hyper-exponential distribution quite well. T…
Multi-access edge computing (MEC) enables placing video content at the edge of the network aiming to improve the quality of experience (QoE) of the mobile clients. Video content caching at edge servers also reduces traffic in the backhaul of the mobile network, hence reducing operational costs for mobile network operators (MNOs). However, minimizing the rate of cache misses and maximizing the average video quality may sometimes be at odds with each other, particularly when the cache size is constrained. Our objective in this article is two fold: First, we explore the impact of fixed video cont…
Dynamic Adaptive Streaming over HTTP(DASH) is becoming the defacto method for effective video traffic delivery at large scale.Its primer success factor returns to the full autonomy given to the streaming clients making them smarter and enabling decentralized logic of video quality decision at granular video chunks following a pull-based paradigm. However,the pure autonomy of the clients inherently results in an overall selfish environment where each client independently strives to improve its Quality of Experience (QoE). Consequently,the clients will hurt each other,including themselves,due to…
The explosion of mobile video traffic imposes tremendous challenges on present cellular networks. To alleviate the pressure on backhaul links and to enhance the quality of experience (QoE) of video streaming service, small cell base stations (SBS) with caching ability are introduced to assist the content delivery. In this paper, we present the first study on the optimal caching strategy of scalable video coding (SVC) streaming in small cell networks with the consideration of channel diversity and video scalability. We formulate an integer programming problem to maximize the average SVC QoE und…
In mobile networks, users may lose coverage when entering a building due to the high signal attenuation at windows and walls. Under such conditions, services with minimum bit-rate requirements, such as video streaming, often show poor Quality-of-Experience (QoE). We will present a Bayesian detector that combines measurements from two Smartphone sensors to decide if a user is inside a building or not. Based on this coverage classification, we will propose an HTTP adaptive streaming (HAS) algorithm to increase playback stability at a high average bitrate. Measurements in a typical office buildin…
We provide a unified framework using which we design scalable dynamic adaptive video streaming algorithms based on index based policies (dubbed DAS-IP) to maximize the Quality of Experience (QoE) provided to clients using video streaming services. Due to the distributed nature of our algorithm, it is easily implementable. We begin by considering the simplest set-up of a one-hop wireless network in which an Access Point (AP) transmits video packets to multiple clients over a shared unreliable channel. The video file meant for each client has been fragmented into several packets, and the server …
Adaptive bitrate streaming (ABR) has become thede factotechnique for videostreaming over the Internet. Despite a flurry of techniques, achieving high quality ABRstreaming over cellular networks remains a tremendous challenge. First, the design ofan ABR scheme needs to balance conflicting Quality of Experience (QoE) metrics suchas video quality, quality changes, stalls and startup performance, which is even harderunder highly dynamic bandwidth in cellular network. Second, streaming providers havebeen moving towards using Variable Bitrate (VBR) encodings for the video content,which introduces ne…
Recently, HTTP-based adaptive streaming has become the de facto standard for video streaming over the Internet. It allows clients to dynamically adapt media characteristics to network conditions in order to ensure a high quality of experience, that is, minimize playback interruptions, while maximizing video quality at a reasonable level of quality changes. In the case of live streaming, this task becomes particularly challenging due to the latency constraints. The challenge further increases if a client uses a wireless network, where the throughput is subject to considerable fluctuations. Cons…
In this paper, we consider the multi-user scheduling problem in millimeter wave (mmWave) video streaming networks, which comprise a streaming server and several users, each requesting a video stream with a different resolution. The main objective is to optimize the long-term average quality of experience (QoE) for all users. We tackle this problem by considering the physical layer characteristics of the mmWave network, including the beam alignment overhead due to pencil-beams. To develop an efficient scheduling policy, we leverage the contextual multi-armed bandit (MAB) models to propose a bea…
Recently, HTTP Adaptive Streaming HAS has received significant attention from both industry and academia based on its ability to enhancing media streaming services over the Internet. Recent research solutions that have tried to improve HAS by adaptation at the client side only may not be completely effective without interacting with routing decisions in the upper layers. In this paper, we address the aforementioned issue by proposing a dynamic bandwidth allocation and management architecture for streaming video flows to improve users satisfaction. We also introduce an initial cross layer hybri…
We take an analytical approach to study the Quality of user Experience (QoE) for video streaming applications. Our propose is to characterize buffer starvations for streaming video with Long-Range-Dependent (LRD) input traffic. Specifically we develop a new analytical framework to investigate Quality of user Experience (QoE) for streaming by considering a Markov Modulated Fluid Model (MMFM) that accurately approximates the Long Range Dependence (LRD) nature of network traffic. We drive the close-form expressions for calculating the distribution of starvation as well as start-up delay using par…
SafeSABR: Risk-Calibrated Adaptive Bitrate Streaming over Starlink Networks
Ningbo Artificial Intelligence Institute, Shanghai Jiao Tong University（宁波人工智能研究院）；School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交大自动化与智能感知学院）；Key Laboratory of System Control and Information Processing, Ministry of Education of China（系统控制与信息处理教育部重点实验室）；Shanghai i-Space Orbital Computing Infrastructure Technology Co., Ltd.（上海i-Space轨道计算基础设施技术有限公司）；Ningbo Industrial Internet Institute（宁波工业互联网研究院）
Proactive tile-based virtual reality (VR) video streaming employs the viewpoint of a user to predict the tiles to be requested, renders and delivers the predicted tiles before playback. Recently, it has been found that the identity and preference of the user can be inferred from the trace of viewpoint uploaded for proactive streaming, which indicates that viewpoint leakage incurs privacy leakage. In this paper, we strive to answer the following questions regarding viewpoint leakage during proactive VR video streaming. When is the viewpoint leaked? Can privacy-preserving approaches (e.g., feder…
360-degree video streaming provides users with immersive experience by letting users determine their field-of-views (FoVs) in real time. To enhance the users' quality of experience (QoE) given their limited bandwidth, recent works have proposed a viewport adaptive 360-degree video streaming model by exploiting the bitrate adaptation in spatial and temporal domains. Under this video streaming model, in this paper, we consider a scenario with a newly generated 360-degree video without viewing history from other users. To maximize the user's QoE, we propose an online bitrate selection algorithm, …
This paper aims to improve video streaming by leveraging a simple observation: users are more sensitive to low quality in certain parts of a video than in others. For instance, rebuffering during key moments of a sports video (e.g., before a goal is scored) is more annoying than rebuffering during normal gameplay. Such dynamic quality sensitivity, however, is rarely captured by current approaches, which predict QoE (quality-of-experience) using one-size-fits-all heuristics that are too simplistic to understand the nuances of video content. Instead of proposing yet another heuristic, we take a …
Most client hosts are equipped with multiple network interfaces (e.g., WiFi and cellular networks). Simultaneous access of multiple interfaces can significantly improve the users' quality of experience (QoE) in video streaming. An intuitive approach to achieve it is to use Multi-path TCP (MPTCP). However, the deployment of MPTCP, especially with link preference, requires OS kernel update at both the client and server side, and a vast amount of commercial content providers do not support MPTCP. Thus, in this paper, we realize a multi-path video streaming algorithm in the application layer inste…
Vehicle-to-everything (V2X) communication is a key enabler that connects vehicles to neighboring vehicles, infrastructure and pedestrians. In the past few years, multimedia services have seen an enormous growth and it is expected to increase as more devices will utilize infotainment services in the future i.e. vehicular devices. Therefore, it is important to focus on user centric measures i.e. quality-of-experience (QoE) such as video quality (resolution) and fluctuations therein. In this paper, a novel joint video quality selection and resource allocation technique is proposed for increasing …
Over the last decade, the bandwidth expansion and MU-MIMO spectral efficiency have promised to increase data throughput by allowing concurrent communication between one Access Point and multiple users. However, we are still a long way from enjoying such MU-MIMO MAC protocol improvements for bandwidth hungry applications such as video streaming in practical WiFi network settings due to heterogeneous channel conditions and devices, unreliable transmissions, and lack of useful feedback exchange among the lower and upper layers' requirements. This paper introduces MuViS, a novel dual-phase optimiz…
Live multimedia streaming from mobile devices is rapidly gaining popularity but little is known about the QoE they provide. In this paper, we examine the Periscope service. We first crawl the service in order to understand its usage patterns. Then, we study the protocols used, the typical quality of experience indicators, such as playback smoothness and latency, video quality, and the energy consumption of the Android application.
Adaptive Bitrate (ABR) Streaming over the cellular networks has been well studied in the literature; however, existing ABR algorithms primarily focus on improving the end-users' Quality of Experience (QoE) while ignoring the resource consumption aspect of the underlying device. Consequently, proactive attempts to download video data to maintain the user's QoE often impact the battery life of the underlying device unless the download attempts are synchronized with the network's channel condition. In this work, we develop EnDASH-5G -- a wrapper over the popular DASH-based ABR streaming algorithm…
We consider the problem of optimizing video delivery for a network supporting video clients streaming stored video. Specifically, we consider the problem of jointly optimizing network resource allocation and video quality adaptation. Our objective is to fairly maximize video clients' Quality of Experience (QoE) realizing tradeoffs among the mean quality, temporal variability in quality, and fairness, incorporating user preferences on rebuffering and cost of video delivery. We present a simple asymptotically optimal online algorithm, NOVA, to solve the problem. NOVA is asynchronous, and using m…
The demand for global video has been burgeoning across industries. With the expansion and improvement of video-streaming services, cloud-based video is evolving into a necessary feature of any successful business for reaching internal and external audiences. This paper considers video streaming over distributed systems where the video segments are encoded using an erasure code for better reliability thus being the first work to our best knowledge that considers video streaming over erasure-coded distributed cloud systems. The download time of each coded chunk of each video segment is character…
In the 360-degree immersive video, a user only views a part of the entire raw video frame based on her viewing direction. However, today's 360-degree video players always fetch the entire panoramic view regardless of users' head movement, leading to significant bandwidth waste that can be potentially avoided. In this paper, we propose a novel adaptive streaming scheme for 360-degree videos. The basic idea is to fetch the invisible portion of a video at the lowest quality based on users' head movement prediction and to adaptively decide the video playback quality for the visible portion based o…
This paper presents a Brownian-approximation framework to optimize the quality of experience (QoE) for real-time video streaming in wireless networks. In real-time video streaming, one major challenge is to tackle the natural tension between the two most critical QoE metrics: playback latency and video interruption. To study this trade-off, we first propose an analytical model that precisely captures all aspects of the playback process of a real-time video stream, including playback latency, video interruptions, and packet dropping. Built on this model, we show that the playback process of a r…
Recent studies into streaming media delivery suggest that performance gains from cache hierarchies such as Information-Centric Networks (ICNs) may be negated by Dynamic Adaptive Streaming (DAS), the de facto method for retrieving multimedia content. The bitrate adaptation mechanisms that drive video streaming clash with caching hierarchies in ways that affect users' Quality of Experience (QoE). Cache performance also diminishes as consumers dynamically select content encoded at different bitrates. In this paper we use the evidence to draw a novel insight: in a cache hierarchy for adaptive stre…
Congestion control protocol and bandwidth allocation problems are often formulated into Network Utility Maximization (NUM) framework. Existing solutions for NUM generally focus on single-layered applications. As applications such as video streaming grow in importance and popularity, addressing user utility function for these multi-layered multimedia applications in NUM formulation becomes vital. In this paper, we propose a new multi-layered user utility model that leverages on studies of human visual perception and quality of experience (QoE) from the fields of computer graphics and human comp…
