Since very few contributions to the development of an unified memory orchestration framework for efficient management of both host and remote idle memory have been made, we present Valet, an efficient approach to orchestration of host and remote shared memory for improving performance of memory intensive workloads. The paper makes three original contributions. First, we redesign the data flow in the critical path by introducing a host-coordinated memory pool that works as a local cache to reduce the latency in the critical path of the host and remote memory orchestration. Second, Valet utilizes unused local memory across containers by managing local memory via Valet host-coordinated memory pool, which allows containers to dynamically expand and shrink their memory allocations according to the workload demands. Third, Valet provides an efficient remote memory reclaiming technique on remote peers, based on two optimizations: (1) an activity-based victim selection scheme to allow the least-active-chunk of data to be selected for serving the eviction requests and (2) a migration protocol to move the least-active-chunk of data to less-memory-pressured remote node. As a result, Valet can effectively reduce the performance impact and migration overhead on local nodes. Our extensive experiments on both NoSQL systems and Machine Learning (ML) workloads show that Valet outperforms existing representative remote paging systems with up to 226X throughput improvement and up to 98% latency decrease over conventional OS swap facility for big data and ML workloads, and by up to 5.5X throughput improvement and up to 78.4% latency decrease over the state-of-the-art remote paging systems. Valet is open sourced at https://github.com/git-disl/Valet.
Virtual machine live migration in cloud environments aims at reducing energy costs and increasing resource utilization. However, its potential has not been fully explored because of simultaneous migrations that may cause user application performance degradation and network congestion. Research efforts on live migration orchestration policies still mostly rely on system level metrics. This work introduces an Application-aware Live Migration Architecture (ALMA) that selects suitable moments for migrations using application characterization data. This characterization consists in recognizing reso…
In this paper we present Foggy, an architectural framework and software platform based on Open Source technologies. Foggy orchestrates application workload, negotiates resources and supports IoT operations for multi-tier, distributed, heterogeneous and decentralized Cloud Computing systems. Foggy is tailored for emerging domains such as 5G Networks and IoT, which demand resources and services to be distributed and located close to data sources and users following the Fog Computing paradigm. Foggy provides a platform for infrastructure owners and tenants (i.e., application providers) offering f…
Containerisation demonstrates its efficiency in application deployment in cloud computing. Containers can encapsulate complex programs with their dependencies in isolated environments, hence are being adopted in HPC clusters. HPC workload managers lack micro-services support and deeply integrated container management, as opposed to container orchestrators (e.g. Kubernetes). We introduce Torque-Operator (a plugin) which serves as a bridge between HPC workload managers and container Orchestrators.
A Robust Adaptive Workload Orchestration in Pure Edge Computing
University College Dublin（都柏林大学）；University Kasdi Merbah（卡斯迪·梅尔巴大学）；Laboratory of Artificial Intelligence and Information Technology (LINATI)（人工智能与信息技术实验室（LINATI））
National University of Singapore（新加坡国立大学）；ByteDance（字节跳动）
Efficient GPU resource scheduling is essential to maximize resource utilization and save training costs for the increasing amount of deep learning workloads in shared GPU clusters. Existing GPU schedulers largely rely on static policies to leverage the performance characteristics of deep learning jobs. However, they can hardly reach optimal efficiency due to the lack of elasticity. To address the problem, we propose ONES, an ONline Evolutionary Scheduler for elastic batch size orchestration. ONES automatically manages the elasticity of each job based on the training batch size, so as to maximi…
Major chip manufacturers have all introduced multicore microprocessors. Multi-socket systems built from these processors are used for running various server applications. However to the best of our knowledge current commercial operating systems are not optimized for multi-threaded workloads running on such servers. Cache-to-cache transfers and remote memory accesses impact the performance of such workloads. This paper presents a unified approach to optimizing OS scheduling algorithms for both cache-to-cache transfers and remote DRAM accesses that also takes cache affinity into account. By obse…
Major chip manufacturers have all introduced multicore microprocessors. Multi-socket systems built from these processors are used for running various server applications. Depending on the application, remote cache-to-cache transfers can severely impact the performance of such workloads. This paper presents a cache optimization that can cut down remote cache-to-cache transfers. By keeping track of remote cache lines loaded from remote caches into last-level-cache and by biasing the cache replacement policy towards such remote cache lines we can reduce the number of cache misses. This in turn re…
Major chip manufacturers have all introduced multicore microprocessors. Multi-socket systems built from these processors are used for running various server applications. Depending on the application that is run on the system, remote memory accesses can impact overall performance. This paper presents a cache optimization that can cut down remote DRAM accesses. By keeping track of remote cache lines loaded from remote DRAM and by biasing the cache replacement policy towards such remote DRAM cache lines the number of cache misses are reduced. This in turn results in improvement of overall perfor…
Multi-socket multi-core servers are used for solving some of the important problems in computing. Remote DRAM accesses can impact performance of certain applications running on such servers. This paper presents a new near linear operating system (OS) scheduling algorithm to reduce the impact of such remote DRAM accesses. By keeping track of the number of local and remote DRAM accesses, using performance counters, for every thread and applying this algorithm, I come up with a new schedule of threads for the next quantum. This new schedule reduces remote DRAM accesses and improves overall perfor…
Major chip manufacturers have all introduced multicore microprocessors. Multi-socket systems built from these processors are routinely used for running various server applications. Depending on the application that is run on the system, remote memory accesses can impact overall performance. This paper presents a new operating system (OS) scheduling optimization to reduce the impact of such remote memory accesses. By observing the pattern of local and remote DRAM accesses for every thread in each scheduling quantum and applying different algorithms, we come up with a new schedule of threads for…
Containers, enabling lightweight environment and performance isolation, fast and flexible deployment, and fine-grained resource sharing, have gained popularity in better application management and deployment in addition to hardware virtualization. They are being widely used by organizations to deploy their increasingly diverse workloads derived from modern-day applications such as web services, big data, and IoT in either proprietary clusters or private and public cloud data centers. This has led to the emergence of container orchestration platforms, which are designed to manage the deployment…
ElastiCo: Elastic Configuration and Interference-Aware Orchestration for GPU Clusters
School of Software, Beihang University（北京航空航天大学软件学院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与技术学院）；School of Computer Science, University of Leeds（利兹大学计算机学院）；School of Computer Science, University of Sydney（悉尼大学计算机学院）
We present RDMAbox, a set of low level RDMA optimizations that provide better performance than previous approaches. The optimizations are packaged in easy-to-use kernel and user space libraries for applications and systems in data center. We demonstrate the flexibility and effectiveness of RDMAbox by implementing a kernel remote paging system and a user space file system using RDMAbox. RDMAbox employs two optimization techniques. First, we suggest RDMA request merging and chaining to further reduce the total number of I/O operations to the RDMA NIC. The I/O merge queue at the same time functio…
SWARM+: Scalable and Resilient Multi-Agent Consensus for Decentralized Data-Aware Workload Management
Renaissance Computing Institute, University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校文艺复兴计算研究所）；Argonne National Laboratory（阿贡国家实验室）；Information Sciences Institute, University of Southern California（南加州大学信息科学研究所）
Massive upgrades to science infrastructure are driving data velocities upwards while stimulating adoption of increasingly data-intensive analytics. While next-generation exascale supercomputers promise strong support for I/O-intensive workflows, HPC remains largely untapped by live experiments, because data transfers and disparate batch-queueing policies are prohibitive when faced with scarce instrument time. To bridge this divide, we introduce Balsam: a distributed orchestration platform enabling workflows at the edge to securely and efficiently trigger analytics tasks across a user-managed f…
Orchestrating centralised service-oriented workflows presents significant scalability challenges that include: the consumption of network bandwidth, degradation of performance, and single points of failure. This paper presents a high-level dataflow specification language that attempts to address these scalability challenges. This language provides simple abstractions for orchestrating large-scale web service workflows, and separates between the workflow logic and its execution. It is based on a data-driven model that permits parallelism to improve the workflow performance. We provide a decentr…
A RAG-Enhanced Bi-Level Cognitive Orchestration Framework for LEO Satellite Networks
Wuhan University of Technology（武汉理工大学）；Hubei Key Laboratory of Transportation Internet of Things, Wuhan University of Technology（武汉理工大学交通物联网湖北省重点实验室）；Swinburne University of Technology（斯威本科技大学）；Hangzhou Dianzi University（杭州电子科技大学）；Technical University of Munich（慕尼黑工业大学）
Traditional data centers are designed with a rigid architecture of fit-for-purpose servers that provision resources beyond the average workload in order to deal with occasional peaks of data. Heterogeneous data centers are pushing towards more cost-efficient architectures with better resource provisioning. In this paper we study the feasibility of using disaggregated architectures for intensive data applications, in contrast to the monolithic approach of server-oriented architectures. Particularly, we have tested a proactive network analysis system in which the workload demands are highly vari…
Cloud workloads today are typically managed in a distributed environment and processed across geographically distributed data centers. Cloud service providers have been distributing data centers globally to reduce operating costs while also improving quality of service by using intelligent workload and resource management strategies. Such large scale and complex orchestration of software workload and hardware resources remains a difficult problem to solve efficiently. Researchers and practitioners have been trying to address this problem by proposing a variety of cloud management techniques. M…
TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling
School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；School of Artificial Intelligence and Data Science, University of Science and Technology of China（中国科学技术大学人工智能与数据科学学院）；Laoshan Laboratory（崂山实验室）；Ocean University of China（中国海洋大学）
Boston University（波士顿大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；The University of Hong Kong（香港大学）；Zhejiang University（浙江大学）；Microsoft（微软）
Deep learning systems have been successfully applied to Euclidean data such as images, video, and audio. In many applications, however, information and their relationships are better expressed with graphs. Graph Convolutional Networks (GCNs) appear to be a promising approach to efficiently learn from graph data structures, having shown advantages in many critical applications. As with other deep learning modalities, hardware acceleration is critical. The challenge is that real-world graphs are often extremely large and unbalanced; this poses significant performance demands and design challenge…
