Recent years have seen Kubernetes emerge as a primary choice for container orchestration. Kubernetes largely targets the cloud environment but new use cases require performant, available and scalable orchestration at the edge. Kubernetes stores all cluster state in etcd, a strongly consistent key-value store. We find that at larger etcd cluster sizes, offering higher availability, write request latency significantly increases and throughput decreases similarly. Coupled with approximately 30% of Kubernetes requests being writes, this directly impacts the request latency and availability of Kubernetes, reducing its suitability for the edge. We revisit the requirement of strong consistency and propose an eventually consistent approach instead. This enables higher performance, availability and scalability whilst still supporting the broad needs of Kubernetes. This aims to make Kubernetes much more suitable for performance-critical, dynamically-scaled edge solutions.
With the emergence of the Internet of Things and 5G technologies, the edge computing paradigm is playing increasingly important roles with better availability, latency-control and performance. However, existing autoscaling tools for edge computing applications do not utilize heterogeneous resources of edge systems efficiently, leaving scope for performance improvement. In this work, we propose a Proactive Pod Autoscaler (PPA) for edge computing applications on Kubernetes. The proposed PPA is able to forecast workloads in advance with multiple user-defined/customized metrics and to scale edge c…
Kubernetes' high resource requirements hamper its adoption in constrained environments such as the edge and fog. Its extensible control plane is a significant contributor to this, consisting of long-lived processes called "controllers" that constantly listen for state changes and use resources even when they are not needed. This paper presents a WebAssembly-based framework for running lightweight controllers on-demand, only when they are needed. This framework extends the WebAssembly System Interface (WASI), in order to run Kubernetes controllers as lightweight Wasm modules. The framework runs…
There is an increasing interest in extending traditional cloud-native technologies, such as Kubernetes, outside the data center to build a continuum towards the edge and between. However, traditional resource orchestration algorithms do not work well in this case, and it is also difficult to test applications for a heterogeneous cloud infrastructure without actually building it. To address these challenges, we propose a new methodology to aid in deploying, testing, and analyzing the effects of microservice placement and scheduling in a heterogeneous Cloud environment. With this methodology, we…
Collaborative edge computing (CEC) is an emerging paradigm enabling sharing of the coupled data, computation, and networking resources among heterogeneous geo-distributed edge nodes. Recently, there has been a trend to orchestrate and schedule containerized application workloads in CEC, while Kubernetes has become the de-facto standard broadly adopted by the industry and academia. However, Kubernetes is not preferable for CEC because its design is not dedicated to edge computing and neglects the unique features of edge nativeness. More specifically, Kubernetes primarily ensures resource provis…
Tianjin University（天津大学）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；Shenzhen University（深圳大学）；the University of British Columbia（不列颠哥伦比亚大学）
Kubernetes (k8s) has the potential to merge the distributed edge and the cloud but lacks a scheduling framework specifically for edge-cloud systems. Besides, the hierarchical distribution of heterogeneous resources and the complex dependencies among requests and resources make the modeling and scheduling of k8s-oriented edge-cloud systems particularly sophisticated. In this paper, we introduce KaiS, a learning-based scheduling framework for such edge-cloud systems to improve the long-term throughput rate of request processing. First, we design a coordinated multi-agent actor-critic algorithm t…
Industries are considering the adoption of cloud computing for real-time applications due to current improvements in network latencies and the advent of Fog and Edge computing. To create an RT-cloud capable of hosting real-time applications, it is increasingly significant to improve the entire stack, including the containerization of applications, and their deployment and orchestration across nodes. However, state-of-the-art orchestrators (e.g., Kubernetes) and underlying container engines are designed for general-purpose applications. They ignore orchestration and management of shared resourc…
Despite the de-facto technological uniformity fostered by the cloud and edge computing paradigms, resource fragmentation across isolated clusters hinders the dynamism in application placement, leading to suboptimal performance and operational complexity. Building upon and extending these paradigms, we propose a novel approach envisioning a transparent continuum of resources and services on top of the underlying fragmented infrastructure, called liquid computing. Fully decentralized, multi-ownership-oriented and intent-driven, it enables an overarching abstraction for improved applications exec…
Currently, due to the advantages of light weight, simple deployment, multi-environment support, short startup time, scalability, and easy migration, container technology has been widely used in both cloud and edge/fog computing, and addresses the problem of device heterogeneity in different computing environments. On this basis, as one of the most popular container orchestration and management systems, Kubernetes almost dominates the cloud environment. However, since it is primarily designed for centralized resource management scenarios where computing resources are sufficient, the system is u…
Towards Scalable Federated Container Orchestration: The CODECO Approach
fortiss GmbH（fortiss有限公司）；Red Hat（红帽）；IBM Research Europe - Zurich（IBM欧洲苏黎世研究中心）；Siemens AG（西门子股份公司）；i2CAT Foundation（i2CAT基金会）；Telefónica（西班牙电信）；Intracom Telecom（Intracom电信）；Democritus University of Thrace / Athena Research Center（色雷斯德谟克利特大学/雅典研究中心）；Universidad Politécnica de Madrid（马德里理工大学）；University of Piraeus Research Centre（比雷埃夫斯大学研究中心）；Netcompany；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）
Edge inference is becoming ever prevalent through its applications from retail to wearable technology. Clusters of networked resource-constrained edge devices are becoming common, yet there is no production-ready orchestration system for deploying deep learning models over such edge networks which adopts the robustness and scalability of the cloud. We present SEIFER, a framework utilizing a standalone Kubernetes cluster to partition a given DNN and place these partitions in a distributed manner across an edge network, with the goal of maximizing inference throughput. The system is node fault-t…
Since emerging edge applications such as Internet of Things (IoT) analytics and augmented reality have tight latency constraints, hardware AI accelerators have been recently proposed to speed up deep neural network (DNN) inference run by these applications. Resource-constrained edge servers and accelerators tend to be multiplexed across multiple IoT applications, introducing the potential for performance interference between latency-sensitive workloads. In this paper, we design analytic models to capture the performance of DNN inference workloads on shared edge accelerators, such as GPU and ed…
The commodity Single Board Computers (SBCs) are increasingly becoming powerful and can execute standard operating systems and mainstream workloads. In the context of cloud-based smart city applications, SBCs can be utilized as Edge computing devices reducing the network communication. In this paper, we investigate the design and implementation of a SBC based edge cluster (SBC-EC) framework for a smart parking application. Since SBCs are resource constrained devices, we devise a container-based framework for a lighter foot-print. Kubernetes was used as an orchestration tool to orchestrate vario…
Conflict-Aware Autonomous Resource Orchestration for 5G Edge Slicing Using Multi-Agent Systems and LLMs
Federal University of Rio Grande do Sul (UFRGS)（南里奥格兰德联邦大学）；Federal University of Viçosa (UFV)（维索卡联邦大学）；University of São Paulo (USP)（圣保罗大学）；University of Minho (UMinho)（米尼奥大学）
Multi-cluster federation is envisioned to be the next-generation cloud infrastructure, where it will play a vital part in the realization of concepts such as edge and fog computing. Orchestrating applications in federated environments poses new challenges to well-known research problems in various fields, such as load-balancing, auto-scaling, resource allocation and service migration. However, as access to real multi-cluster infrastructure is limited, a test-bed that provides similar characteristics to a real system is in demand. To enable researchers in associated fields to quickly setup expe…
