Fog computing promises to enable machine learning tasks to scale to large amounts of data by distributing processing across connected devices. Two key challenges to achieving this goal are heterogeneity in devices compute resources and topology constraints on which devices can communicate with each other. We address these challenges by developing the first network-aware distributed learning optimization methodology where devices optimally share local data processing and send their learnt parameters to a server for aggregation at certain time intervals. Unlike traditional federated learning frameworks, our method enables devices to offload their data processing tasks to each other, with these decisions determined through a convex data transfer optimization problem that trades off costs associated with devices processing, offloading, and discarding data points. We analytically characterize the optimal data transfer solution for different fog network topologies, showing for example that the value of offloading is approximately linear in the range of computing costs in the network. Our subsequent experiments on testbed datasets we collect confirm that our algorithms are able to improve network resource utilization substantially without sacrificing the accuracy of the learned model. In these experiments, we also study the effect of network dynamics, quantifying the impact of nodes entering or exiting the network on model learning and resource costs.
Stateful applications and virtualized network functions (VNFs) can benefit from state externalization to increase their reliability, scalability, and inter-operability. To keep and share the externalized state, distributed data stores (DDSs) are a powerful tool allowing for the management of classical trade-offs in consistency, availability and partitioning tolerance. With the advent of Fog and Edge Computing, stateful applications and VNFs are pushed from the data centers toward the network edge. This poses new challenges on DDSs that are tailored to a deployment in Cloud data centers. In thi…
Distributed fog and edge applications communicate over unreliable networks and are subject to high communication delays. This makes using existing distributed coordination technologies from cloud applications infeasible, as they are built on the assumption of a highly reliable, low-latency datacenter network to achieve strict consistency with low overheads. To help implement configuration and state management for fog platforms and applications, we propose a novel decentralized approach that lets systems specify coordination strategies and membership for different sets of coordination data.
Fog computing emerged as a crucial platform for the deployment of IoT applications. The complexity of such applications requires methods that handle the resource diversity and network structure of Fog devices while maximizing the service placement and reducing resource wastage. Prior studies in this domain primarily focused on optimizing specific application requirements and fail to address the network topology combined with the different types of resources encountered in Fog devices. To overcome these problems, we propose a multilayer resource-aware partitioning method to minimize the resourc…
Mobile clients that consume and produce data are abundant in fog environments. Low latency access to this data can only be achieved by storing it in close physical proximity to the clients. Current data store systems fall short as they do not replicate data based on client movement. We propose an approach to predictive replica placement that autonomously and proactively replicates data close to likely client locations.
Fog computing is a promising computing paradigm for time-sensitive Internet of Things (IoT) applications. It helps to process data close to the users, in order to deliver faster processing outcomes than the Cloud; it also helps to reduce network traffic. The computation environment in the Fog computing is highly dynamic and most of the Fog devices are battery powered hence the chances of application failure is high which leads to delaying the application outcome. On the other hand, if we rerun the application in other devices after the failure it will not comply with time-sensitiveness. To sol…
As the Internet of Things (IoT) becomes a part of our daily life, there is a rapid growth in connected devices. A well-established approach based on cloud computing technologies cannot provide the necessary quality of service in such an environment, particularly in terms of reducing data latency. Today, fog computing technology is seen as a novel approach for processing large amounts of critical and time-sensitive data. This article reviews cloud computing technology and analyzes the prerequisites for the evolution of this approach and the emergence of the concept of fog computing. As part of …
Resource management in Fog computing is very complicated as it engages significant number of diverse and resource constraint Fog nodes to meet computational demand of IoT-enabled systems in distributed manner. Its integration with Cloud triggers further difficulties in combined resource management. Different sensing frequency of IoT devices, distributed application structure and their coordination also influence resource management in Fog computing environment. For advancement of Fog and its resource management, the necessity of extensive research in beyond question. To develop and evaluate di…
Fog computing is a promising computing paradigm in which IoT data can be processed near the edge to support time-sensitive applications. However, the availability of the resources in the computation device is not stable since they may not be exclusively dedicated to the processing in the Fog environment. This, combined with dynamic user behaviour, can affect the execution of applications. To address dynamic changes in user behaviour in a resource limited Fog device, this paper proposes a Multi-Criteria-based resource allocation policy with resource reservation in order to minimise overall dela…
Internet of Things typically involves a significant number of smart sensors sensing information from the environment and sharing it to a cloud service for processing. Various architectural abstractions, such as Fog and Edge computing, have been proposed to localize some of the processing near the sensors and away from the central cloud servers. In this paper, we propose Edge-Fog Cloud which distributes task processing on the participating cloud resources in the network. We develop the Least Processing Cost First (LPCF) method for assigning the processing tasks to nodes which provide the optima…
Fog computing is an emerging paradigm that aims to meet the increasing computation demands arising from the billions of devices connected to the Internet. Offloading services of an application from the Cloud to the edge of the network can improve the overall Quality-of-Service (QoS) of the application since it can process data closer to user devices. Diverse Fog nodes ranging from Wi-Fi routers to mini-clouds with varying resource capabilities makes it challenging to determine which services of an application need to be offloaded. In this paper, a context-aware mechanism for distributing appli…
Mobile clients that consume and produce data are abundant in fog environments and low latency access to this data can only be achieved by storing it in their close physical proximity. To adapt data replication in fog data stores in an efficient manner and make client data available at the fog node that is closest to the client, the systems need to predict both client movement and pauses in data consumption. In this paper, we present variations of Markov model algorithms that can run on clients to increase the data availability while minimizing excess data. In a simulation, we find the availabi…
Despite constant improvements in efficiency, today's data centers and networks consume enormous amounts of energy and this demand is expected to rise even further. An important research question is whether and how fog computing can curb this trend. As real-life deployments of fog infrastructure are still rare, a significant part of research relies on simulations. However, existing power models usually only target particular components such as compute nodes or battery-constrained edge devices. Combining analytical and discrete-event modeling, we develop a holistic but granular energy consumptio…
In this paper we present Foggy, an architectural framework and software platform based on Open Source technologies. Foggy orchestrates application workload, negotiates resources and supports IoT operations for multi-tier, distributed, heterogeneous and decentralized Cloud Computing systems. Foggy is tailored for emerging domains such as 5G Networks and IoT, which demand resources and services to be distributed and located close to data sources and users following the Fog Computing paradigm. Foggy provides a platform for infrastructure owners and tenants (i.e., application providers) offering f…
Edge and Fog computing paradigms overcome the limitations of cloud-centric execution for different latency-sensitive Internet of Things (IoT) applications by offering computing resources closer to the data sources. Small single-board computers (SBCs) like Raspberry Pis (RPis) are widely used as computing nodes in both paradigms. These devices are usually equipped with moderate speed processors and provide support for peripheral interfacing and networking, making them well-suited to deal with IoT-driven operations such as data sensing, analysis, and actuation. However, these small Edge devices …
Smart farming is a recent innovation in the agriculture sector that can improve the agricultural yield by using smarter, automated, and data driven farm processes that interact with IoT devices deployed on farms. A cloud-fog infrastructure provides an effective platform to execute IoT applications. While fog computing satisfies the real-time processing need of delay-sensitive IoT services by bringing virtualized services closer to the IoT devices, cloud computing allows execution of applications with higher computational requirements. The deployment of IoT applications is a critical challenge …
Performance and Security Aware Distributed Service Placement in Fog Computing
The Faculty of Information Technology, Monash University（蒙纳士大学信息技术学院）；School of Computer Science and Engineering, The University of New South Wales（新南威尔士大学计算机科学与工程系）；The Quantum Cloud Computing and Distributed Systems (qCLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息学系量子云计算与分布式系统实验室）
Excessive tail end-to-end latency occurs with conventional message brokers as a result of having massive numbers of geographically distributed devices communicate through a message broker. On the other hand, broker-less messaging systems, though ensure low latency, are highly dependent on the limitation of direct device-to-device (D2D) communication technologies, and cannot scale well as large numbers of resource-limited devices exchange messages. In this paper, we propose FogMQ, a cloud-based message broker system that overcomes the limitations of conventional systems by enabling autonomous d…
Agentic Fog: A Policy-driven Framework for Distributed Intelligence in Fog Computing
Sarhad University of Science & Information Technology（萨赫德科学与技术大学）；GIK Institute of Engineering Sciences & Technology（吉克工程科学与技术学院）；University of Essex（埃塞克斯大学）
Federated learning refers to conducting training on multiple distributed devices and collecting model weights from them to derive a shared machine-learning model. This allows the model to get benefit from a rich source of data available from multiple sites. Also, since only model weights are collected from distributed devices, the privacy of those data is protected. It is useful in a situation where collaborative training of machine learning models is necessary while training data are highly sensitive. This study aims at investigating the implementation of lightweight federated learning to be …
Tactile Internet is an emerging concept that focuses on supporting high-fidelity, ultra-responsive, and widely available human-to-machine interactions. To reduce the transmission latency and alleviate Internet congestion, fog computing has been advocated as an important component of the Tactile Internet. In this paper, we focus on energy-efficient design of fog computing networks that support low-latency Tactile Internet applications. We investigate two performance metrics: Service response time of end-users and power usage efficiency of fog nodes. We quantify the fundamental tradeoff between …
The combination of edge and cloud in the fog computing paradigm enables a new breed of data-intensive applications. These applications, however, have to face a number of fog-specific challenges which developers have to repetitively address for every single application. In this paper, we propose a replication service specifically tailored to the needs of data-intensive fog applications that aims to ease or eliminate challenges caused by the highly distributed and heterogeneous environment fog applications operate in. Furthermore, we present our prototypical proof-of-concept implementation FBase…
During the last decade, Cloud computing has efficiently exploited the economy of scale by providing low cost computational and storage resources over the Internet, eventually leading to consolidation of computing resources into large data centers. However, the nascent of the highly decentralized Internet of Things (IoT) technologies that cannot effectively utilize the centralized Cloud infrastructures pushes computing towards resource dispersion. Fog computing extends the Cloud paradigm by enabling dispersion of the computational and storage resources at the edge of the network in a close prox…
Fog computing is a new computational paradigm that emerged from the need to reduce network usage and latency in the Internet of Things (IoT). Fog can be considered as a continuum between the cloud layer and IoT users that allows the execution of applications or storage/processing of data in network infrastructure devices. The heterogeneity and wider distribution of fog devices are the key differences between cloud and fog infrastructure. Genetic-based optimization is commonly used in distributed systems; however, the differentiating features of fog computing require new designs, studies, and e…
Redundancy is abundant in Fog networks (i.e., many computing and storage points) and grows linearly with network size. We demonstrate the transformational role of coding in Fog computing for leveraging such redundancy to substantially reduce the bandwidth consumption and latency of computing. In particular, we discuss two recently proposed coding concepts, namely Minimum Bandwidth Codes and Minimum Latency Codes, and illustrate their impacts in Fog computing. We also review a unified coding framework that includes the above two coding techniques as special cases, and enables a tradeoff between…
Resource management is the principal factor to fully utilize the potential of Edge/Fog computing to execute real-time and critical IoT applications. Although some resource management frameworks exist, the majority are not designed based on distributed containerized components. Hence, they are not suitable for highly distributed and heterogeneous computing environments. Containerized resource management frameworks such as FogBus2 enable efficient distribution of framework's components alongside IoT applications' components. However, the management, deployment, health-check, and scalability of a…
Fog computing is a distributed paradigm that provides computational resources in the users' vicinity. Fog orchestration is a set of functionalities that coordinate the dynamic infrastructure and manage the services to guarantee the Service Level Agreements. Monitoring is an orchestration functionality of prime importance. It is the basis for resource management actions, collecting status of resource and service and delivering updated data to the orchestrator. There are several cloud monitoring solutions and tools, but none of them comply with fog characteristics and challenges. Fog monitoring …
Fog computing, as a distributed paradigm, offers cloud-like services at the edge of the network with low latency and high-access bandwidth to support a diverse range of IoT application scenarios. To fully utilize the potential of this computing paradigm, scalable, adaptive, and accurate scheduling mechanisms and algorithms are required to efficiently capture the dynamics and requirements of users, IoT applications, environmental properties, and optimization targets. This paper presents a taxonomy of recent literature on scheduling IoT applications in Fog computing. Based on our new classificat…
In recent years, the number of Internet of Things (IoT) devices/sensors has increased to a great extent. To support the computational demand of real-time latency-sensitive applications of largely geo-distributed IoT devices/sensors, a new computing paradigm named "Fog computing" has been introduced. Generally, Fog computing resides closer to the IoT devices/sensors and extends the Cloud-based computing, storage and networking facilities. In this chapter, we comprehensively analyse the challenges in Fogs acting as an intermediate layer between IoT devices/ sensors and Cloud datacentres and revi…
We present FLIC, a distributed software data caching framework for fogs that reduces network traffic and latency. FLICis targeted toward city-scale deployments of cooperative IoT devices in which each node gathers and shares data with surrounding devices. As machine learning and other data processing techniques that require large volumes of training data are ported to low-cost and low-power IoT systems, we expect that data analysis will be moved away from the cloud. Separation from the cloud will reduce reliance on power-hungry centralized cloud-based infrastructure. However, city-scale deploy…
With the pervasiveness of IoT devices, smart-phones and improvement of location-tracking technologies huge volume of heterogeneous geo-tagged (location specific) data is generated which facilitates several location-aware services. The analytics with this spatio-temporal (having location and time dimensions) datasets provide varied important services such as, smart transportation, emergency services (health-care, national defence or urban planning). While cloud paradigm is suitable for the capability of storage and computation, the major bottleneck is network connectivity loss. In time-critical…
Intending to support new emerging applications with latency requirements below what can be offered by the cloud data centers, the edge and fog computing paradigms have reared. In such systems, the real-time instant data is processed closer to the edge of the network, instead of the remote data centers. With the advances in edge and fog computing systems, novel and efficient solutions based on Distributed Hash Tables (DHTs) emerge and play critical roles in system design. Several DHT-based solutions have been proposed to either augment the scalability and efficiency of edge and fog computing in…
Fog/Edge computing model allows harnessing of resources in the proximity of the Internet of Things (IoT) devices to support various types of real-time IoT applications. However, due to the mobility of users and a wide range of IoT applications with different requirements, it is a challenging issue to satisfy these applications' requirements. The execution of IoT applications exclusively on one fog/edge server may not be always feasible due to limited resources, while execution of IoT applications on different servers needs further collaboration among servers. Also, considering user mobility, s…
Fog/Edge computing is a novel computing paradigm supporting resource-constrained Internet of Things (IoT) devices by the placement of their tasks on the edge and/or cloud servers. Recently, several Deep Reinforcement Learning (DRL)-based placement techniques have been proposed in fog/edge computing environments, which are only suitable for centralized setups. The training of well-performed DRL agents requires manifold training data while obtaining training data is costly. Hence, these centralized DRL-based techniques lack generalizability and quick adaptability, thus failing to efficiently tac…
In this paper, we intend to reduce the operational cost of cloud data centers with the help of fog devices, which can avoid the revenue loss due to wide-area network propagation delay and save network bandwidth cost by serving nearby cloud users. Since fog devices may not be owned by a cloud service provider, they should be compensated for serving the requests of cloud users. When taking economical compensation into consideration, the optimal number of requests processed locally by each fog device should be decided. As a result, existing load balancing schemes developed for cloud data centers …
Fog computing brings the functionality of the cloud near the edge of the network with the help of fog devices/micro data centers ($mdcs$). Job scheduling in such systems is a complex problem due to the hierarchical and geo-distributed nature of fog devices. We propose two fog scheduling algorithms, named $FiFSA$ (Hierarchical $Fi$rst $F$og $S$cheduling $A$lgorithm) and $EFSA$ ( Hierarchical $E$lected $F$og $S$cheduling $A$lgorithm). We consider a hierarchical model of fog devices, where the computation power of fog devices present in higher tiers is greater than those present in lower tiers. H…
We propose a novel integrated fog cloud IoT (IFCIoT) architectural paradigm that promises increased performance, energy efficiency, reduced latency, quicker response time, scalability, and better localized accuracy for future IoT applications. The fog nodes (e.g., edge servers, smart routers, base stations) receive computation offloading requests and sensed data from various IoT devices. To enhance performance, energy efficiency, and real-time responsiveness of applications, we propose a reconfigurable and layered fog node (edge server) architecture that analyzes the applications' characterist…
The Internet of Everything (IoE) solutions gradually bring every object online, and processing data in centralized cloud does not scale to requirements of such environment. This is because, there are applications such as health monitoring and emergency response that require low latency and delay caused by transferring data to the cloud and then back to the application can seriously impact the performance. To this end, Fog computing has emerged, where cloud computing is extended to the edge of the network to decrease the latency and network congestion. Fog computing is a paradigm for managing a…
For effective use of edge computing in an IoT application, we need to partition the application into tasks and map them into the cloud, fog (edge server), device levels such that the resources at the different levels are optimally used to meet the overall quality of service requirements. In this paper, we consider four concerns about application-to-fog mapping: task placement at different levels, data filtering to limit network loading, fog fail-over, and data consistency, and reacting to hotspots at the edge. We describe a programming language and middleware we created for edge computing that…
Industry 4.0 becomes possible through the convergence between Operational and Information Technologies. All the requirements to realize the convergence is integrated on the Fog Platform. Fog Platform is introduced between the cloud server and edge devices when the unprecedented generation of data causes the burden of the cloud server, leading the ineligible latency. In this new paradigm, we divide the computation tasks and push it down to edge devices. Furthermore, local computing (at edge side) may improve privacy and trust. To address these problems, we present a new method, in which we deco…
Purdue University（普渡大学）；NC State University（北卡罗来纳州立大学）
Machine learning (ML) tasks are becoming ubiquitous in today's network applications. Federated learning has emerged recently as a technique for training ML models at the network edge by leveraging processing capabilities across the nodes that collect the data. There are several challenges with employing conventional federated learning in contemporary networks, due to the significant heterogeneity in compute and communication capabilities that exist across devices. To address this, we advocate a new learning paradigm called fog learning which will intelligently distribute ML model training acro…
With the rapid growth of the Internet of Things (IoT) and a wide range of mobile devices, the conventional cloud computing paradigm faces significant challenges (high latency, bandwidth cost, etc.). Motivated by those constraints and concerns for the future of the IoT, modern architectures are gearing toward distributing the cloud computational resources to remote locations where most end-devices are located. Edge and fog computing are considered as the key enablers for applications where centralized cloud-based solutions are not suitable. In this paper, we review the high-level definition of …
Emerging technologies like the Internet of Things (IoT) require latency-aware computation for real-time application processing. In IoT environments, connected things generate a huge amount of data, which are generally referred to as big data. Data generated from IoT devices are generally processed in a cloud infrastructure because of the on-demand services and scalability features of the cloud computing paradigm. However, processing IoT application requests on the cloud exclusively is not an efficient solution for some IoT applications, especially time-sensitive ones. To address this issue, Fo…
Internet of Things (IoT) is leading to the pervasive availability of streaming data about the physical world, coupled with edge computing infrastructure deployed as part of smart cities and 5G rollout. These constrained, less reliable but cheap resources are complemented by fog resources that offer federated management and accelerated computing, and pay-as-you-go cloud resources. There is a lack of intuitive means to deploy application pipelines to consume such diverse streams, and to execute them reliably on edge and fog resources. We propose an innovative application model to declaratively s…
