In this paper, we propose a load balancing algorithm based on Reinforcement Learning (RL) to optimize the performance of Fog Computing for real-time IoT applications. The algorithm aims to minimize the waiting delay of IoT workloads in dynamic environments with unpredictable traffic demands, using intelligent workload distribution. Unlike previous studies, our solution does not require load and resource information from Fog nodes to preserve the privacy of service providers, who may wish to hide such information to prevent competitors from calculating better pricing strategies. The proposed algorithm is evaluated on a Discrete-event Simulator (DES) to mimic practical deployment in real environments, and its generalization ability is tested on simulations longer than what it was trained on. Our results show that our proposed approach outperforms baseline load balancing methods under different workload generation rates, while ensuring the privacy of Fog service providers. Furthermore, the environment representation we proposed for the RL agent demonstrates better performance compared to the commonly used representations for RL solutions in the literature, which compromise privacy.
The powerful paradigm of Fog computing is currently receiving major interest, as it provides the possibility to integrate virtualized servers into networks and brings cloud service closer to end devices. To support this distributed intelligent platform, Software-Defined Network (SDN) has emerged as a viable network technology in the Fog computing environment. However, uncertainties related to task demands and the different computing capacities of Fog nodes, inquire an effective load balancing algorithm. In this paper, the load balancing problem has been addressed under the constraint of achiev…
Recently, fog computing has been introduced as a modern distributed paradigm and complement to cloud computing to provide services. Fog system extends storing and computing to the edge of the network, which can solve the problem about service computing of the delay-sensitive applications remarkably besides enabling the location awareness and mobility support. Load balancing is an important aspect of fog networks that avoids a situation with some under-loaded or overloaded fog nodes. Quality of Service (QoS) parameters such as resource utilization, throughput, cost, response time, performance, …
In this paper, we intend to reduce the operational cost of cloud data centers with the help of fog devices, which can avoid the revenue loss due to wide-area network propagation delay and save network bandwidth cost by serving nearby cloud users. Since fog devices may not be owned by a cloud service provider, they should be compensated for serving the requests of cloud users. When taking economical compensation into consideration, the optimal number of requests processed locally by each fog device should be decided. As a result, existing load balancing schemes developed for cloud data centers …
The Internet of Things (IoT) requires a new processing paradigm that inherits the scalability of the cloud while minimizing network latency using resources closer to the network edge. Building up such flexibility within the edge-to-cloud continuum consisting of a distributed networked ecosystem of heterogeneous computing resources is challenging. Furthermore, IoT traffic dynamics and the rising demand for low-latency services foster the need for minimizing the response time and balanced service placement. Load-balancing for fog computing becomes a cornerstone for cost-effective system manageme…
Parallel iterative applications often suffer from load imbalance, one of the most critical performance degradation factors. Hence, load balancing techniques are used to distribute the workload evenly to maximize performance. A key challenge is to know \textit{when} to use load balancing techniques. In general, this is done through load balancing criteria, which trigger load balancing based on runtime application data and/or user-defined information. In the first part of this paper, we introduce a novel, automatic load balancing criterion derived from a simple mathematical model. In the second …
We propose a GPU fine-grained load-balancing abstraction that decouples load balancing from work processing and aims to support both static and dynamic schedules with a programmable interface to implement new load-balancing schedules. Prior to our work, the only way to unleash the GPU's potential on irregular problems has been to workload-balance through application-specific, tightly coupled load-balancing techniques. With our open-source framework for load-balancing, we hope to improve programmers' productivity when developing irregular-parallel algorithms on the GPU, and also improve the ove…
Data load balancing is a challenging task in the P2P systems. Distributed hash table (DHT) abstraction, heterogeneous nodes, and non uniform distribution of objects are the reasons to cause load imbalance in structured P2P overlay networks. Previous works solved the load balancing problem by assuming the homogeneous capabilities of nodes, unawareness of the link latency during transferring load, and imposing logical structures to collect and reassign load. We propose a distributed load balancing algorithm with the topology awareness feature by using the concept of virtual servers. In our appro…
This document presents an analysis of different load balance strategies for a Plasma physics code that models high energy particle beams with PIC method. A comparison of different load balancing algorithms is given: static or dynamic ones. Lagrangian and Eulerian partitioning techniques have been investigated.
A parallel computer system is a collection of processing elements that communicate and cooperate to solve large computational problems efficiently. To achieve this, at first the large computational problem is partitioned into several tasks with different work-loads and then are assigned to the different processing elements for computation. Distribution of the work load is known as Load Balancing. An appropriate distribution of work-loads across the various processing elements is very important as disproportional workloads can eliminate the performance benefit of parallelizing the job. Hence, l…
Cloud Computing is a new trend emerging in IT environment with huge requirements of infrastructure and resources. Load Balancing is an important aspect of cloud computing environment. Efficient load balancing scheme ensures efficient resource utilization by provisioning of resources to cloud users on demand basis in pay as you say manner. Load Balancing may even support prioritizing users by applying appropriate scheduling criteria. This paper presents various load balancing schemes in different cloud environment based on requirements specified in Service Level Agreement (SLA).
In cloud environments, load balancing task scheduling is an important issue that directly affects resource utilization. Unquestionably, load balancing scheduling is a serious aspect that must be considered in the cloud research field due to the significant impact on both the back end and front end. Whenever an effective load balance has been achieved in the cloud, then good resource utilization will also be achieved. An effective load balance means distributing the submitted workload over cloud VMs in a balanced way, leading to high resource utilization and high user satisfaction. In this pape…
In parallel computing, a problem is divided into a set of smaller tasks that are distributed across multiple processing elements. Balancing the load of the processing elements is key to achieving good performance and scalability. If the computational costs of the individual tasks vary over time in an unpredictable way, dynamic load balancing aims at migrating them between processing elements so as to maintain load balance. During dynamic load balancing, the tasks amount to indivisible work packets with a real-valued cost. For this case of indivisible, real- valued loads, we analyze the balanci…
Effective utilization of Multiple-Instruction-Multiple-Data (MIMD) parallel computers requires the application of good load balancing techniques. In this paper we show that heuristics derived from observation of complex natural systems, such as the mammalian immune system, can lead to effective load balancing strategies. In particular, the immune system processes of regulation, suppression, tolerance, and memory are seen to be powerful load balancing mechanisms. We provide a detailed example of our approach applied to parallelization of an image processing task, that of extracting the circuit …
Load balancing is prevalent in practical application (e.g., web) deployments seen today. One such load balancer, HAProxy, remains relevant as an open-source, easy-to-use system. In the context of web systems, the load balancer tier possesses significant influence over system performance and the incurred cost, which is decisive for cloud-based deployments. Therefore, it is imperative to properly tune the load balancer configuration and get the most performance out of the existing resources. In this technical report, we first introduce the HAProxy architecture and its load balancing methods. The…
This work studies distributed algorithms for locally optimal load-balancing: We are given a graph of maximum degree $Δ$, and each node has up to $L$ units of load. The task is to distribute the load more evenly so that the loads of adjacent nodes differ by at most $1$. If the graph is a path ($Δ= 2$), it is easy to solve the fractional version of the problem in $O(L)$ communication rounds, independently of the number of nodes. We show that this is tight, and we show that it is possible to solve also the discrete version of the problem in $O(L)$ rounds in paths. For the general case ($Δ> 2$), w…
In this paper, we introduce DLB, a Deep Learning based load Balancing mechanism, to effectively address the data skew problem. The key idea of DLB is to replace hash functions in the load balancing mechanisms with deep learning models, which are trained to be able to map different distributions of workloads and data to the servers in a uniform manner. We implemented DLB and deployed it on a practical Cloud environment using CloudSim. Experimental results using both synthetic and real-world data sets show that compared with traditional hash function-based load balancing methods, DLB is able to …
The effectiveness and scalability of MapReduce-based implementations of complex data-intensive tasks depend on an even redistribution of data between map and reduce tasks. In the presence of skewed data, sophisticated redistribution approaches thus become necessary to achieve load balancing among all reduce tasks to be executed in parallel. For the complex problem of entity resolution, we propose and evaluate two approaches for such skew handling and load balancing. The approaches support blocking techniques to reduce the search space of entity resolution, utilize a preprocessing MapReduce job…
Load balancing is vital for the efficient and long-term operation of cloud data centers. With virtualization, post (reactive) migration of virtual machines after allocation is the traditional way for load balancing and consolidation. However, reactive migration is not easy to obtain predefined load balance objectives and may interrupt services and bring instability. Therefore, we provide a new approach, called Prepartition, for load balancing. It partitions a VM request into a few sub-requests sequentially with start time, end time and capacity demands, and treats each sub-request as a regular…
The use of under-utilized Internet resources is widely recognized as a viable form of high performance computing. Sustained processing power of roughly 40T FLOPS using 4 million volunteered Internet hosts has been reported for embarrassingly parallel problems. At the same time, peer-to-peer (P2P) file sharing networks, with more than 50 million participants, have demonstrated the capacity for scale in distributed systems. This paper contributes a study of load balancing techniques for a general class of loosely-synchronous parallel algorithms when executed over a P2P network. We show that dece…
CMT-nek is a new scientific application for performing high fidelity predictive simulations of particle laden explosively dispersed turbulent flows. CMT-nek involves detailed simulations, is compute intensive and is targeted to be deployed on exascale platforms. The moving particles are the main source of load imbalance as the application is executed on parallel processors. In a demonstration problem, all the particles are initially in a closed container until a detonation occurs and the particles move apart. If all processors get an equal share of the fluid domain, then only some of the proce…
Big data analytics in cloud environments introduces challenges such as real-time load balancing besides security, privacy, and energy efficiency. In this paper, we propose a novel load balancing algorithm in cloud environments that performs resource allocation and task scheduling efficiently. The proposed load balancer reduces the execution response time in big data applications performed on clouds. Scheduling, in general, is an NP-hard problem. In our proposed algorithm, we provide solutions to reduce the search area that leads to reduced complexity of the load balancing. We recommend two mat…
For the parallel computation of partial differential equations, one key is the grid partitioning. It requires that each process owns the same amount of computations, and also, the partitioning quality should be proper to reduce the communications among processes. When calculating the partial differential equations using adaptive finite element methods, the grid and the basis functions adjust in each iteration, which introduce load balancing issues. The grid should be redistributed dynamically. This paper studies dynamic load balancing algorithms and the implementation on the adaptive finite el…
Heavy traffic analysis for load balancing policies has relied heavily on the condition of state-space collapse onto a single-dimensional line in previous works. In this paper, via Lyapunov-drift analysis, we rigorously prove that even under a multi-dimensional state-space collapse, steady-state heavy-traffic delay optimality can still be achieved for a general load balancing system. This result directly implies that achieving steady-state heavy-traffic delay optimality simply requires that no server is idling while others are busy at heavy loads, thus complementing and extending the result obt…
The problem of minimizing mean response time of generic jobs submitted to a heterogenous distributed computer systems is considered in this paper. A static load balancing strategy, in which decision of redistribution of loads does not depend on the state of the system, is used for this purpose. The article is closely related to a previous article on the same topic. The present article points out number of inconsistencies in the previous article, provides a new formulation, and discusses the impact of new findings, based on the improved formulation, on the results of the previous article.
We study the problem of load balancing in distributed stream processing engines, which is exacerbated in the presence of skew. We introduce Partial Key Grouping (PKG), a new stream partitioning scheme that adapts the classical "power of two choices" to a distributed streaming setting by leveraging two novel techniques: key splitting and local load estimation. In so doing, it achieves better load balancing than key grouping while being more scalable than shuffle grouping. We test PKG on several large datasets, both real-world and synthetic. Compared to standard hashing, PKG reduces the load imb…
Load balancing is the process of improving the Performance of a parallel and distributed system through is distribution of load among the processors [1-2]. Most of the previous work in load balancing and distributed decision making in general, do not effectively take into account the uncertainty and inconsistency in state information but in fuzzy logic, we have advantage of using crisps inputs. In this paper, we present a new approach for implementing dynamic load balancing algorithm with fuzzy logic, which can face to uncertainty and inconsistency of previous algorithms, further more our algo…
With the rapid development in wide area networks and low cost, powerful computational resources, grid computing has gained its popularity. With the advent of grid computing, space limitations of conventional distributed systems can be overcome and underutilized computing resources at different locations around the world can be put to distributed jobs. Workload and resource management is the main key grid services at the service level of grid infrastructures, out of which load balancing in the main concern for grid developers. It has been found that load is the major problem which server faces,…
We present an open-source topology-aware hierarchical unstructured mesh partitioning and load-balancing tool TreePart. The framework provides powerful abstractions to automatically detect and build hierarchical MPI topology resembling the hardware at runtime. Using this information it intelligently chooses between shared and distributed parallel algorithms for partitioning and load-balancing. It provides a range of partitioning methods by interfacing with existing shared and distributed memory parallel partitioning libraries. It provides powerful and scalable abstractions like one-sided distri…
We consider the following load balancing process for $m$ tokens distributed arbitrarily among $n$ nodes connected by a complete graph: In each time step a pair of nodes is selected uniformly at random. Let $\ell_1$ and $\ell_2$ be their respective number of tokens. The two nodes exchange tokens such that they have $\lceil(\ell_1 + \ell_2)/2\rceil$ and $\lfloor(\ell_1 + \ell_2)/2\rfloor$ tokens, respectively. We provide a simple analysis showing that this process reaches almost perfect balance within $O(n\log{n} + n \logΔ)$ steps, where $Δ$ is the maximal initial load difference between any two…
E-science applications may require huge amounts of data and high processing power where grid infrastructures are very suitable for meeting these requirements. The load distribution in a grid may vary leading to the bottlenecks and overloaded sites. We describe a hierarchical dynamic load balancing protocol for Grids. The Grid consists of clusters and each cluster is represented by a coordinator. Each coordinator first attempts to balance the load in its cluster and if this fails, communicates with the other coordinators to perform transfer or reception of load. This process is repeated periodi…
Equation systems resulting from a p-version FEM discretisation typically require a special treatment as iterative solvers are not very efficient here. Applying hierarchical concepts based on a nested dissection approach allow for both the design of sophisticated solvers as well as for advanced parallelisation strategies. To fully exploit the underlying computing power of parallel systems, dynamic load balancing strategies become an essential component.
As compute power increases with time, more involved and larger simulations become possible. However, it gets increasingly difficult to efficiently use the provided computational resources. Especially in particle-based simulations with a spatial domain partitioning large load imbalances can occur due to the simulation being dynamic. Then a static domain partitioning may not be suitable. This can deteriorate the overall runtime of the simulation significantly. Sophisticated load balancing strategies must be designed to alleviate this problem. In this paper we conduct a systematic evaluation of t…
Acceleration of graph applications on GPUs has found large interest due to the ubiquitous use of graph processing in various domains. The inherent \textit{irregularity} in graph applications leads to several challenges for parallelization. A key challenge, which we address in this paper, is that of load-imbalance. If the work-assignment to threads uses node-based graph partitioning, it can result in skewed task-distribution, leading to poor load-balance. In contrast, if the work-assignment uses edge-based graph partitioning, the load-balancing is better, but the memory requirement is relativel…
This paper presents Fast load balancing technique inspired by Bioinformatics is a special case to assign a particular patient with a specialist physician cluster at real time. The work is considered soft presentation of the Gaussian mixture model based on the extracted features supplied by patients. Based on the likelihood ratio test, the patient is assigned to a specialist physician cluster. The presented algorithms efficiently handle any size and any numbers of incoming patient requests and rapidly placed them to the specialist physician cluster. Hence it smoothly balances the traffic load o…
Load balancing algorithms play critical roles in systems where the workload has to be distributed across multiple resources, such as cores in multiprocessor system, computers in distributed computing, and network links. In this paper, we study and evaluate four load balancing methods: random, round robin, shortest queue, and shortest queue with stale load information. We build a simulation model and compare mean delay of the systems for the load balancing methods. We also provide a method to improve shortest queue with stale load information load balancing. A performance analysis for the impro…
The classification of the most used load balancing algorithms in distributed systems (including cloud technology, cluster systems, grid systems) is described. Comparative analysis of types of the load balancing algorithms is conducted in accordance with the classification, the advantages and drawbags of each type of the algorithms are shown. Performance indicators characterizing each algorithm are indicated.
In the load balancing problem, each node in a network is assigned a load, and the goal is to equally distribute the loads among the nodes, by preforming local load exchanges. While load balancing was extensively studied in static networks, only recently a load balancing algorithm for dynamic networks with a bounded convergence time was presented. In this paper, we further study the time complexity of load balancing in the context of dynamic networks. First, we show that randomness is not necessary, and present a deterministic algorithm which slightly improves the running time of the previous a…
With technological advancements and constant changes of Internet, cloud computing has been today's trend. With the lower cost and convenience of cloud computing services, users have increasingly put their Web resources and information in the cloud environment. The availability and reliability of the client systems will become increasingly important. Today cloud applications slightest interruption, the impact will be significant for users. It is an important issue that how to ensure reliability and stability of the cloud sites. Load balancing would be one good solution. This paper presents a fr…
Fine-grained workload and resource balancing is the key to high performance for regular and irregular computations on the GPUs. In this dissertation, we conduct an extensive survey of existing load-balancing techniques to build an abstraction that addresses the difficulty of scheduling computations on the GPU. We propose a GPU fine-grained load-balancing abstraction that decouples load balancing from work processing and aims to support both static and dynamic schedules with a programmable interface to implement new load-balancing schedules. Prior to our work, the only way to unleash the GPU's …
This paper studies the problem of load-balancing the demand for content in a peer-to-peer network across heterogeneous peer nodes that hold replicas of the content. Previous decentralized load balancing techniques in distributed systems base their decisions on periodic updates containing information about load or available capacity observed at the serving entities. We show that these techniques do not work well in the peer-to-peer context; either they do not address peer node heterogeneity, or they suffer from significant load oscillations. We propose a new decentralized algorithm, Max-Cap, ba…
This article proposes a design and implementation of a low cost two-tier architecture model for high availability cluster combined with load-balancing and shared storage technology to achieve desired scale of three-tier architecture for application load balancing e.g. web servers. The research work proposes a design that physically omits Network File System (NFS) server nodes and implements NFS server functionalities within the cluster nodes, through Red Hat Cluster Suite (RHCS) with High Availability (HA) proxy load balancing technologies. In order to achieve a low-cost implementation in term…
Current-day data centers and high-volume cloud services employ a broad set of heterogeneous servers. In such settings, client requests typically arrive at multiple entry points, and dispatching them to servers is an urgent distributed systems problem. This paper presents an efficient solution to the load balancing problem in such systems that improves on and overcomes problems of previous solutions. The load balancing problem is formulated as a stochastic optimization problem, and an efficient algorithmic solution is obtained based on a subtle mathematical analysis of the problem. Finally, ext…
