With the edge computing becoming an increasingly adopted concept in system architectures, it is expected its utilization will be additionally heightened when combined with deep learning (DL) techniques. The idea behind integrating demanding processing algorithms in Internet of Things (IoT) and edge devices, such as Deep Neural Network (DNN), has in large measure benefited from the development of edge computing hardware, as well as from adapting the algorithms for use in resource constrained IoT devices. Surprisingly, there are no models yet to optimally place and use machine learning in edge computing. In this paper, we propose the first model of optimal placement of Deep Neural Network (DNN) Placement and Inference in edge computing. We present a mathematical formulation to the DNN Model Variant Selection and Placement (MVSP) problem considering the inference latency of different model-variants, communication latency between nodes, and utilization cost of edge computing nodes. We evaluate our model numerically, and show that for low load increasing model co-location decreases the average latency by 33% of millisecond-scale per request, and for high load, by 21%.
Edge inference has become more widespread, as its diverse applications range from retail to wearable technology. Clusters of networked resource-constrained edge devices are becoming common, yet no system exists to split a DNN across these clusters while maximizing the inference throughput of the system. We present an algorithm which partitions DNNs and distributes them across a set of edge devices with the goal of minimizing the bottleneck latency and therefore maximizing inference throughput. The system scales well to systems of different node memory capacities and numbers of nodes. We find t…
This work evaluates three Fog Computing dataplacement algorithms via experiments carried out with theiFogSim simulator. The paper describes the three algorithms(Cloud-only, Mapping, Edge-ward) in the context of an Internetof Things scenario, which has been based on an e-Health systemwith variations in applications and network topology. Resultsachieved show that edge placement strategies are beneficial toassist cloud computing in lowering latency and cloud energyexpenditure.
Fog computing extends cloud computing technology to the edge of the infrastructure to let IoT applications access objects' data with reduced latency, location awareness and dynamic computation. By displacing workloads from the central cloud to the edge devices, fog computing overcomes communication bottlenecks avoiding raw data transfer to the central cloud, thus paving the way for the next generation IoT-based applications. In this paper we study scheduling and placement of applications in fog computing, which is key to ensure profitability for the involved stakeholders. We consider a scenari…
The deployment of edge computing infrastructure requires a careful placement of the edge servers, with an aim to improve application latencies and reduce data transfer load in opportunistic Internet of Things systems. In the edge server placement, it is important to consider computing capacity, available deployment budget, and hardware requirements for the edge servers and the underlying backbone network topology. In this paper, we thoroughly survey the existing literature in edge server placement, identify gaps and present an extensive set of parameters to be considered. We then develop a nov…
Cloud computing is a revolutionary process that has impacted the manner of using networks. It allows a high level of flexibility as Virtual Machines (VMs) run elastically workloads on physical machines in data centers. The issue of placing virtual machines (VMP) in cloud environments is an important challenge that has been thoroughly addressed, although not yet completely resolved. This article discusses the different problems that may disrupt the placement of VMs and Virtual Network Functions (VNFs), and classifies the existing solutions into five major objective functions based on multiple p…
Ubiquitous artificial intelligence (AI) is considered one of the key services in 6G systems. AI services typically rely on deep neural network (DNN) requiring heavy computation. Hence, in order to support ubiquitous AI, it is crucial to provide a solution for offloading or distributing computational burden due to DNN, especially at end devices with limited resources. We develop an optimization framework for assigning the computation tasks of DNN inference jobs to computing resources in the network, so as to reduce the inference latency. To this end, we propose a layered graph model with which …
The proliferation in data volume and processing requests calls for a new breed of on-demand computing. Fog computing is proposed to address the limitations of cloud computing by extending processing and storage resources to the edge of the network. Cloud and fog computing employ virtual machines (VMs) for efficient resource utilization. In order to optimize the virtual environment, VMs can be migrated or replicated over geo-distributed physical machines for load balancing and energy efficiency. In this work, we investigate the offloading of VM services from the cloud to the fog considering the…
In mobile edge computing (MEC) systems, edge service caching refers to pre-storing the necessary programs for executing computation tasks at MEC servers. At resource-constrained edge servers, service caching placement is in general a complicated problem that highly correlates to the offloading decisions of computation tasks. In this paper, we consider a single edge server that assists a mobile user (MU) in executing a sequence of computation tasks. In particular, the MU can run its customized programs at the edge server, while the server can selectively cache the previously generated programs …
Caching high-frequency reuse contents at the edge servers in the mobile edge computing (MEC) network omits the part of backhaul transmission and further releases the pressure of data traffic. However, how to efficiently decide the caching contents for edge servers is still an open problem, which refers to the cache capacity of edge servers, the popularity of each content, and the wireless channel quality during transmission. In this paper, we discuss the influence of unknown user density and popularity of content on the cache placement solution at the edge server. Specifically, towards the imp…
Vehicle-to-everything (V2X) services are attracting a lot of attention in the research and industry communities due to their applicability in the landscape of connected and autonomous vehicles. Such applications have stringent performance requirements in terms of complex data processing and low latency communications which are utilized to ensure road safety and improve road conditions. To address these challenges, the placement of V2X applications through leveraging of edge computing paradigm, that distributes the computing capabilities to access points in proximity to the vehicles, presents i…
The proliferation of innovative mobile services such as augmented reality, networked gaming, and autonomous driving has spurred a growing need for low-latency access to computing resources that cannot be met solely by existing centralized cloud systems. Mobile Edge Computing (MEC) is expected to be an effective solution to meet the demand for low-latency services by enabling the execution of computing tasks at the network-periphery, in proximity to end-users. While a number of recent studies have addressed the problem of determining the execution of service tasks and the routing of user reques…
Satellite edge computing has become a promising way to provide computing services for Internet of Things (IoT) devices in remote areas, which are out of the coverage of terrestrial networks, nevertheless, it is not suitable for large-scale IoT devices due to the resource limitation of satellites. Cloud computing can provide sufficient available resources for IoT devices but it does not meet the service requirements of delay sensitive users as high network latency. Collaborative edge and cloud computing is to facilitate flexible service provisioning for numerous IoT devices by incorporating the…
The increasing popularity of cloud computing has resulted in a proliferation of data centers. Effective placement of data centers improves network performance and minimizes clients' perceived latency. The problem of determining the optimal placement of data centers in a large network is a classical uncapacitated $k$-median problem. Traditional works have focused on centralized algorithms, which requires knowledge of the overall network topology and information about the customers' service demands. Moreover, centralized algorithms are computationally expensive and do not scale well with the siz…
Satellite networks, as a supplement to terrestrial networks, can provide effective computing services for Internet of Things (IoT) users in remote areas. Due to the resource limitation of satellites, such as in computing, storage, and energy, a computation task from a IoT user can be divided into several parts and cooperatively accomplished by multiple satellites to improve the overall operational efficiency of satellite networks. Network function virtualization (NFV) is viewed as a new paradigm in allocating network resources on-demand. Satellite edge computing combined with the NFV technolog…
Edge computing hosts applications close to the end users and enables low-latency real-time applications. Modern applications inturn have adopted the microservices architecture which composes applications as loosely coupled smaller components, or services. This complements edge computing infrastructure that are often resource constrained and may not handle monolithic applications. Instead, edge servers can independently deploy application service components, although at the cost of communication overheads. Consistently meeting application service level objectives while also optimizing applicati…
Fog computing is an emerging paradigm that aims to improve the efficiency and QoS of cloud computing by extending the cloud to the edge of the network. This paper develops a comprehensive energy efficiency analysis framework based on mathematical modeling and heuristics to study the offloading of virtual machine (VM) services from the cloud to the fog. The analysis addresses the impact of different factors including the traffic between the VM and its users, the VM workload, the workload versus number of users profile and the proximity of fog nodes to users. Overall, the power consumption can b…
Leveraging recent advances on mobile edge computing (MEC), edge intelligence has emerged as a promising paradigm to support mobile artificial intelligence (AI) applications at the network edge. In this paper, we consider the AI service placement problem in a multi-user MEC system, where the access point (AP) places the most up-to-date AI program at user devices to enable local computing/task execution at the user side. To fully utilize the stringent wireless spectrum and edge computing resources, the AP sends the AI service program to a user only when enabling local computing at the user yield…
With the advent of large LEO satellite communication networks to provide global broadband Internet access, interest in providing edge computing resources within LEO networks has emerged. The LEO Edge promises low-latency, high-bandwidth access to compute and storage resources for a global base of clients and IoT devices regardless of their geographical location. Current proposals assume compute resources or service replicas at every LEO satellite, which requires high upfront investments and can lead to over-provisioning. To implement and use the LEO Edge efficiently, methods for server and ser…
With constrained resources, what, where, and how to cache at the edge is one of the key challenges for edge computing systems. The cached items include not only the application data contents but also the local caching of edge services that handle incoming requests. However, current systems separate the contents and services without considering the latency interplay of caching and queueing. Therefore, in this paper, we propose a novel class of stochastic models that enable the optimization of content caching and service placement decisions jointly. We first explain how to apply layered queueing…
Deep Neural Network (DNN) is becoming adopted for video analytics on mobile devices. To reduce the delay of running DNNs, many mobile devices are equipped with Neural Processing Units (NPU). However, due to the resource limitations of NPU, these DNNs have to be compressed to increase the processing speed at the cost of accuracy. To address the low accuracy problem, we propose a Confidence Based Offloading (CBO) framework for deep learning video analytics. The major challenge is to determine when to return the NPU classification result based on the confidence level of running the DNN, and when …
Recent years have witnessed a remarkable development in communication and computing systems, mainly driven by the increasing demands of data and processing intensive applications such as virtual reality, M2M, connected vehicles, IoT services, to name a few. Massive amounts of data will be collected by various mobile and fixed terminals that will need to be processed in order to extract knowledge from the data. Traditionally, a centralized approach is taken for processing the collected data using large data centers connected to a core network. However, due to the scale of the Internet-connected…
Content Delivery Networks (CDNs) have gained immense popularity over the years. Replica server placement is a key design issue in CDNs. It entails placing replica servers at meticulous locations, such that cost is minimized and Quality of Service (QoS) of end-users is satisfied. Many replica server placement models have been proposed in the literature of traditional CDN. As the CDN architecture is evolving through the adoption of emerging paradigms, such as, cloud computing and Network Functions Virtualization (NFV), new algorithms are being proposed. In this paper, we present a comprehensive …
Mobile edge computing is a new cloud computing paradigm which makes use of small-sized edge-clouds to provide real-time services to users. These mobile edge-clouds (MECs) are located in close proximity to users, thus enabling users to seamlessly access applications running on MECs. Due to the co-existence of the core (centralized) cloud, users, and one or multiple layers of MECs, an important problem is to decide where (on which computational entity) to place different components of an application. This problem, known as the application or workload placement problem, is notoriously hard, and t…
Softwarization and virtualization are key concepts for emerging industries that require ultra-low latency. This is only possible if computing resources, traditionally centralized at the core of communication networks, are moved closer to the user, to the network edge. However, the realization of Edge Computing (EC) in the sixth generation (6G) of mobile networks requires efficient resource allocation mechanisms for the placement of the Virtual Network Functions (VNFs). Machine learning (ML) methods, and more specifically, Reinforcement Learning (RL), are a promising approach to solve this prob…
Network function Virtualization (NFV) and Mobile Edge Computing (MEC) are promising 5G technologies to support resource-demanding mobile applications. In NFV, one must process the service function chain (SFC) in which a set of network functions must be executed in a specific order. Moreover, the MEC technology enables computation offloading of service requests from mobile users to remote servers to potentially reduce energy consumption and processing delay for the mobile application. This paper considers the optimization of the computation offloading, resource allocation, and SFC placement in …
Mobile edge computing (MEC) is emerging to support delay-sensitive 5G applications at the edge of mobile networks. When a user moves erratically among multiple MEC nodes, the challenge of how to dynamically migrate its service to maintain service performance (i.e., user-perceived latency) arises. However, frequent service migration can significantly increase operational cost, incurring the conflict between improving performance and reducing cost. To address these mis-aligned objectives, this paper studies the performance optimization of mobile edge service placement under the constraint of lon…
We propose a cloudlet network architecture to bring the computing resources from the centralized cloud to the edge. Thus, each User Equipment (UE) can communicate with its Avatar, a software clone located in a cloudlet, with lower end-to-end (E2E) delay. However, UEs are moving over time, and so the low E2E delay may not be maintained if UEs' Avatars stay in their original cloudlets. Thus, live Avatar migration (i.e., migrating a UE's Avatar to a suitable cloudlet based on the UE's location) is enabled to maintain low E2E delay between each UE and its Avatar. On the other hand, the migration i…
Sun Yat-sen University（中山大学）；University of Nevada, Reno（内华达大学里诺分校）；Arizona State University（亚利桑那州立大学）
Recent advances in artificial intelligence have driven increasing intelligent applications at the network edge, such as smart home, smart factory, and smart city. To deploy computationally intensive Deep Neural Networks (DNNs) on resource-constrained edge devices, traditional approaches have relied on either offloading workload to the remote cloud or optimizing computation at the end device locally. However, the cloud-assisted approaches suffer from the unreliable and delay-significant wide-area network, and the local computing approaches are limited by the constrained computing capability. To…
As a key technology of enabling Artificial Intelligence (AI) applications in 5G era, Deep Neural Networks (DNNs) have quickly attracted widespread attention. However, it is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources. What's worse, traditional cloud-assisted DNN inference is heavily hindered by the significant wide-area network latency, leading to poor real-time performance as well as low quality of user experience. To address these challenges, in this paper, we propose Edgent, a framework that leverages edge computing for…
Nowadays, deep neural networks (DNNs) are the core enablers for many emerging edge AI applications. Conventional approaches to training DNNs are generally implemented at central servers or cloud centers for centralized learning, which is typically time-consuming and resource-demanding due to the transmission of a large amount of data samples from the device to the remote cloud. To overcome these disadvantages, we consider accelerating the learning process of DNNs on the Mobile-Edge-Cloud Computing (MECC) paradigm. In this paper, we propose HierTrain, a hierarchical edge AI learning framework, …
University of Miami（迈阿密大学）；University of California, Riverside（加州大学河滨分校）；Huazhong University of Science and Technology（华中科技大学）
Shared edge computing platforms deployed at the radio access network are expected to significantly improve quality of service delivered by Application Service Providers (ASPs) in a flexible and economic way. However, placing edge service in every possible edge site by an ASP is practically infeasible due to the ASP's prohibitive budget requirement. In this paper, we investigate the edge service placement problem of an ASP under a limited budget, where the ASP dynamically rents computing/storage resources in edge sites to host its applications in close proximity to end users. Since the benefit …
School of Data and Computer Science, Sun Yat-sen University（中山大学数据科学与计算机学院）
Mobile edge computing is a new computing paradigm, which pushes cloud computing capabilities away from the centralized cloud to the network edge. However, with the sinking of computing capabilities, the new challenge incurred by user mobility arises: since end-users typically move erratically, the services should be dynamically migrated among multiple edges to maintain the service performance, i.e., user-perceived latency. Tackling this problem is non-trivial since frequent service migration would greatly increase the operational cost. To address this challenge in terms of the performance-cost…
Deploying V2X services has become a challenging task. This is mainly due to the fact that such services have strict latency requirements. To meet these requirements, one potential solution is adopting mobile edge computing (MEC). However, this presents new challenges including how to find a cost efficient placement that meets other requirements such as latency. In this work, the problem of cost-optimal V2X service placement (CO-VSP) in a distributed cloud/edge environment is formulated. Additionally, a cost-focused delay-aware V2X service placement (DA-VSP) heuristic algorithm is proposed. Sim…
Deep neural networks (DNNs) sustain high performance in today's data processing applications. DNN inference is resource-intensive thus is difficult to fit into a mobile device. An alternative is to offload the DNN inference to a cloud server. However, such an approach requires heavy raw data transmission between the mobile device and the cloud server, which is not suitable for mission-critical and privacy-sensitive applications such as autopilot. To solve this problem, recent advances unleash DNN services using the edge computing paradigm. The existing approaches split a DNN into two parts and…
