Function as a Service (FaaS) permits cloud customers to deploy to cloud individual functions, in contrast to complete virtual machines or Linux containers. All major cloud providers offer FaaS products (Amazon Lambda, Google Cloud Functions, Azure Serverless); there are also popular open-source implementations (Apache OpenWhisk) with commercial offerings (Adobe I/O Runtime, IBM Cloud Functions). A new feature of FaaS is function composition: a function may (sequentially) call another function, which, in turn, may call yet another function - forming a chain of invocations. From the perspective of the infrastructure, a composed FaaS is less opaque than a virtual machine or a container. We show that this additional information enables the infrastructure to reduce the response latency. In particular, knowing the sequence of future invocations, the infrastructure can schedule these invocations along with environment preparation. We model resource management in FaaS as a scheduling problem combining (1) sequencing of invocations, (2) deploying execution environments on machines, and (3) allocating invocations to deployed environments. For each aspect, we propose heuristics. We explore their performance by simulation on a range of synthetic workloads. Our results show that if the setup times are long compared to invocation times, algorithms that use information about the composition of functions consistently outperform greedy, myopic algorithms, leading to significant decrease in response latency.
Scheduling of service requests in Cloud computing has traditionally focused on the reduction of pre-service wait, generally termed as waiting time. Under certain conditions such as peak load, however, it is not always possible to give reasonable response times to all users. This work explores the fact that different users may have their own levels of tolerance or patience with response delays. We introduce scheduling strategies that produce better assignment plans by prioritising requests from users who expect to receive the results earlier and by postponing servicing jobs from those who are m…
In this paper, we study the market-oriented online bi-objective service scheduling problem for pleasingly parallel jobs with variable resources in cloud environments, from the perspective of SaaS (Software-as-as-Service) providers who provide job-execution services. The main process of scheduling SaaS services in clouds is: a SaaS provider purchases cloud instances from IaaS providers to schedule end users' jobs and charges users accordingly. This problem has several particular features, such as the job-oriented end users, the pleasingly parallel jobs with soft deadline constraints, the online…
Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware. In this work, we systematically analyze the opportunities and challenges of providing multi-tenant deep learning services on the general-purpose CPU architecture from the aspects of scheduling granularity and code gene…
Modern latency-critical online services often rely on composing results from a large number of server components. Hence the tail latency (e.g. the 99th percentile of response time), rather than the average, of these components determines the overall service performance. When hosted on a cloud environment, the components of a service typically co-locate with short batch jobs to increase machine utilizations, and share and contend resources such as caches and I/O bandwidths with them. The highly dynamic nature of batch jobs in terms of their workload types and input sizes causes continuously cha…
Mobile edge computing (MEC) emerges as a promising solution for servicing delay-sensitive tasks at the edge network. A body of recent literature started to focus on cost-efficient service placement and request scheduling. This work investigates the joint optimization of service placement and request scheduling in a dense MEC network, and develops an efficient online algorithm that achieves close-to-optimal performance. Our online algorithm consists of two basic modules: (1) a regularization with look-ahead approach from competitive online convex optimization, for decomposing the offline relaxe…
Companies are rushing to deliver their services and solutions through the cloud. The scheduling process is very critical in reducing delays. Scheduling also has a role in accessing resources without excessive waiting time. All this in context of modern advances in infrastructure and the emergence of Blockchain-as-a-service. What if integration is done between a hybrid scheduling algorithm and blockchain technology via the cloud. This integration aims to enhance and provide the service uninterruptedly. This method is distinguished, compared to other scheduling algorithms such as shortest-job-fi…
Co-location and memory sharing between latency-critical services, such as key-value store and web search, and best-effort batch jobs is an appealing approach to improving memory utilization in multi-tenant datacenter systems. However, we find that the very diverse goals of job co-location and the GNU/Linux system stack can lead to severe performance degradation of latency-critical services under memory pressure in a multi-tenant system. We address memory pressure for latency-critical services via fast memory allocation and proactive reclamation. We find that memory allocation latency dominates…
Given the network latency variability observed in data centers, applications' performance is also determined by their placement within the data centre. We present NoMora, a cluster scheduling architecture whose core is represented by a latency-driven, application performance-aware, cluster scheduling policy. The policy places the tasks of an application taking into account the expected performance based on the measured network latency between pairs of hosts in the data center. Furthermore, if a tenant's application experiences increased network latency, and thus lower application performance, …
Cloud computing environments often have to deal with random-arrival computational workloads that vary in resource requirements and demand high Quality of Service (QoS) obligations. It is typical that a Service-Level-Agreement (SLA) is employed to govern the QoS obligations of the cloud computing service provider to the client. A typical challenge service-providers face every day is maintaining a balance between the limited resources available for computing and the high QoS requirements of varying random demands. Any imbalance in managing these conflicting objectives may result in either dissat…
Fast, reliable, and efficient data transmission across wide-area networks is a predominant bottleneck for data-intensive cloud applications. This paper introduces OneDataShare, which is designed to eliminate the issues plaguing effective cloud-based data transfers of varying file sizes and across incompatible transfer end-points. The vision of OneDataShare is to achieve high-speed data communication, interoperability between multiple transfer protocols, and accurate estimation of delivery time for advance planning, thereby maximizing user-profit through improved and faster data analysis for bu…
Cloud Computing is a paradigm of both parallel processing and distributed computing. It offers computing facilities as a utility service in pay as par use manner. Virtualization, self service provisioning, elasticity and pay per use are the key features of Cloud Computing. It provides different types of resources over the Internet to perform user submitted tasks. In cloud environment, huge number of tasks are executed simultaneously, an effective Task Scheduling is required to gain better performance of the cloud system. Various Cloud Based Task Scheduling algorithms are available that schedul…
Vehicular cloud (VC) platforms integrate heterogeneous and distributed resources of moving vehicles to offer timely and cost-effective computing services. However, the dynamic nature of VCs (i.e., limited contact duration among vehicles), caused by vehicles' mobility, poses unique challenges to the execution of computation-intensive applications/tasks with directed acyclic graph (DAG) structure, where each task consists of multiple interdependent components (subtasks). In this paper, we study scheduling of DAG tasks over dynamic VCs, where multiple subtasks of a DAG task are dispersed across v…
League Championship Algorithm (LCA) is a sports-inspired population based algorithmic framework for global optimization over a continuous search space first proposed by Ali Husseinzadeh Kashan in the year 2009. A common characteristic between all population based optimization algorithms similar to the LCA is that, they attemt to move a population of achievable solutions to potential areas of the search space during optimization. In this paper, we proposed a job scheduling algorithm based on the L CA optimization technique for the infrastructure as a service (IaaS) cloud. Three other establishe…
Dynamic Service Scheduling and Resource Management in Energy-Harvesting Multi-access Edge Computing
University of Leeds（利兹大学）；Southern University of Science and Technology (SUSTech)（南方科技大学）；University of Thessaly（色萨利大学）；Research Institute of Trustworthy Autonomous Systems (RITAS), SUSTech（南方科技大学可信自主系统研究院）
Cloud computing has attracted both end-users and Cloud Service Providers (CSPs) in recent years. Improving resource utilization rate (RUtR), such as CPU and memory usages on servers, while maintaining Quality-of-Service (QoS) is one key challenge faced by CSPs with warehouse-scale data centers. Prior works proposed various algorithms to reduce energy cost or to improve RUtR, which either lack the fine-grained task scheduling capabilities, or fail to take a comprehensive system model into consideration. This article presents H2O-Cloud, a Hierarchical and Hybrid Online task scheduling framework …
Modern latency-critical online services such as search engines often process requests by consulting large input data spanning massive parallel components. Hence the tail latency of these components determines the service latency. To trade off result accuracy for tail latency reduction, existing techniques use the components responding before a specified deadline to produce approximate results. However, they may skip a large proportion of components when load gets heavier, thus incurring large accuracy losses. This paper presents AccuracyTrader that produces approximate results with small accur…
Latency-critical services have been widely deployed in cloud environments. For cost-efficiency, multiple services are usually co-located on a server. Thus, run-time resource scheduling becomes the pivot for QoS control in these complicated co-location cases. However, the scheduling exploration space enlarges rapidly with the increasing server resources, making the schedulers hardly provide ideal solutions quickly. More importantly, we observe that there are "resource cliffs" in the scheduling exploration space. They affect the exploration efficiency and always lead to severe QoS fluctuations. …
The University of Tokyo（东京大学）；Preferred Networks, Inc.（优选网络公司）
One significant challenge in the job scheduling of computing clusters for the development of deep learning algorithms is the efficient scheduling of trial-and-error (TE) job, the type of job in which the users seek to conduct small-scale experiments while monitoring their processes. Unfortunately, the existing job schedulers to date do not feature well-balanced scheduling for the mixture of TE jobs and best-effort (BE) jobs, or they can handle the mixture in limited situations at most. To fill in this niche, we propose an algorithm that can significantly reduce the latency of TE jobs in versat…
Cloud computing distributes computing tasks across numerous distributed resources for large-scale calculation. The task scheduling problem is a long-standing problem in cloud-computing services with the purpose of determining the quality, availability, reliability, and ability of the cloud computing. This paper is an extension and a correction to our previous conference paper entitled Multi Objective Scheduling in Cloud Computing Using MOSSO published in 2018 IEEE Congress on Evolutionary Computation. More new algorithms, testing, and comparisons have been implemented to solve the bi-objective…
Community networks (CNs) have seen an increase in the last fifteen years. Their members contact nodes which operate Internet proxies, web servers, user file storage and video streaming services, to name a few. Detecting communities of nodes with properties (such as co-location) and assessing node eligibility for service placement is thus a key-factor in optimizing the experience of users. We present a novel solution for the problem of service placement as a two-phase approach, based on: 1) community finding using a scalable graph label propagation technique and 2) a decentralized election proc…
TS-EoH: An Edge Server Task Scheduling Algorithm Based on Evolution of Heuristic
School of Computer Science Central China Normal University Wuhan, China yatong\；School of Computer Science Central China Normal University Wuhan, China
This paper considers the problem of service placement and task scheduling on a three-tiered edge-to-cloud platform when user requests must be met by a certain deadline. Time-sensitive applications (e.g., augmented reality, gaming, real-time video analysis) have tight constraints that must be met. With multiple possible computation centers, the "where" and "when" of solving these requests becomes paramount when meeting their deadlines. We formulate the problem of meeting users' deadlines while minimizing the total cost of the edge-to-cloud service provider as an Integer Linear Programming (ILP)…
In Software-Defined Networking (SDN)-enabled cloud data centers, live migration is a key approach used for the reallocation of Virtual Machines (VMs) in cloud services and Virtual Network Functions (VNFs) in Service Function Chaining (SFC). Using live migration methods, cloud providers can address their dynamic resource management and fault tolerance objectives without interrupting the service of users. However, in cloud data centers, performing multiple live migrations in arbitrary order can lead to service degradation. Therefore, efficient migration planning is essential to reduce the impact…
Queuing systems with redundant requests have drawn great attention because of their promise to reduce the job completion time and variability. Despite a large body of work on the topic, we are still far from fully understanding the benefits of redundancy in practice. We here take one step towards practical systems by studying queuing systems with bi-modal job service time distribution. Such distributions have been observed in practice, as can be seen in, e.g., Google cluster traces. We develop an analogy to a classical urns and balls problem, and use it to study the queuing time performance of…
Accelerating AIGC Services with Latent Action Diffusion Scheduling in Edge Networks
BNU-HKBU United International College（北师港浸会联合国际学院）；Hong Kong Baptist University（香港浸会大学）；Institute of Artificial Intelligence and Future Networks, Beijing Normal University（北京师范大学人工智能与未来网络研究院）
To amortize cost, cloud vendors providing DNN acceleration as a service to end-users employ consolidation and virtualization to share the underlying resources among multiple DNN service requests. This paper makes a case for a "preemptible" neural processing unit (NPU) and a "predictive" multi-task scheduler to meet the latency demands of high-priority inference while maintaining high throughput. We evaluate both the mechanisms that enable NPUs to be preemptible and the policies that utilize them to meet scheduling objectives. We show that preemptive NPU multi-tasking can achieve an average 7.8…
We study the performance of non-adaptive scheduling policies in computing systems with multiple servers. Compute jobs are mostly regular, with modest service requirements. However, there are sporadic data intensive jobs, whose expected service time is much higher than that of the regular jobs. Forthis model, we are interested in the effect of scheduling policieson the average time a job spends in the system. To this end, we introduce two performance indicators in a simplified, only-arrival system. We believe that these performance indicators are good predictors of the relative performance of t…
Cloud-computing shares a common pool of resources across customers at a scale that is orders of magnitude larger than traditional multi-user systems. Constituent physical compute servers are allocated multiple "virtual machines" (VM) to serve simultaneously. Each VM user should ideally be unaffected by others' demand. Naturally, this environment produces new challenges for the service providers in meeting customer expectations while extracting an efficient utilization from server resources. We study a new cloud service metric that measures prolonged latency or delay suffered by customers. We m…
