As a prominent network abstraction, coflow models efficiently capture communication patterns in data centers. Since coflow scheduling in large-scale data centers is $\mathcal{NP}$-hard, the existing literature has predominantly focused on limited environments with $m=2$ network cores, relying on flow splitting, which introduces substantial operational overhead. Crucially, no approximation algorithm with provable performance guarantees has been proposed for the more practical, non-splitting coflow scheduling problem, even for the $m=2$ case, let alone for general hybrid architectures. To bridge this critical gap, this paper investigates the non-splitting problem within a hybrid, heterogeneous parallel network featuring multiple network cores ($m \ge 2$) composed of Electronic Packet Switches (EPS), not-all-stop Optical Circuit Switches (OCS), and all-stop OCS. We propose three unified polynomial-time approximation algorithms that minimize the makespan and the total weighted coflow completion time across this hybrid environment without incurring any splitting overhead. Let $τ$ denote the maximum flow degree across all ports in the network, and let $m$ be the number of network cores. To minimize the makespan, our algorithm achieves an approximation ratio of $2\min\left\{2τ-1, m+τ-1\right\}$ in the hybrid architecture. To minimize the total weighted coflow completion time, our algorithm achieves an approximation ratio of $16\min\left\{2τ-1, 2m+τ-1\right\}$ in the hybrid architecture. Moreover, we characterize the approximation ratios of our algorithm under different architectural combinations.
Coflow is a recently proposed network abstraction to capture communication patterns in data centers. The coflow scheduling problem in large data centers is one of the most important $NP$-hard problems. Previous research on coflow scheduling focused mainly on the single-switch model. However, with recent technological developments, this single-core model is no longer sufficient. This paper considers the coflow scheduling problem in identical parallel networks. The identical parallel network is an architecture based on multiple network cores running in parallel. Coflow can be considered as divis…
Coflow is a recently proposed network abstraction for data-parallel computing applications. This paper considers scheduling coflows with precedence constraints in identical parallel networks, such as to minimize the total weighted completion time of coflows. The identical parallel network is an architecture based on multiple network cores running in parallel. In the divisible coflow scheduling problem, the proposed algorithm achieves $(6-\frac{2}{m})μ$ and $(5-\frac{2}{m})μ$ approximate ratios for arbitrary release time and zero release time, respectively, where $m$ is the number of network co…
Coflow scheduling models communication requests in parallel computing frameworks where multiple data flows between shared resources need to be completed before computation can continue. In this paper, we introduce Path-based Coflow Scheduling, a generalized problem variant that considers coflows as collections of flows along fixed paths on general network topologies with node capacity restrictions. For this problem, we minimize the coflows' total weighted completion time. We show that flows on paths in the original network can be interpreted as hyperedges in a hypergraph and transform the path…
Efficient Approximation Algorithms for Scheduling Coflows with Precedence Constraints in Identical Parallel Networks to Minimize Weighted Completion Time
The coflow scheduling problem is considered: given an input/output switch with each port having a fixed capacity, find a scheduling algorithm that minimizes the weighted sum of the coflow completion times respecting the port capacities, where each flow of a coflow has a demand per input/output port, and coflow completion time is the finishing time of the last flow of the coflow. The objective of this paper is to present theoretical guarantees on approximating the sum of coflow completion time in the non-clairvoyant setting, where on a coflow arrival, only the number of flows, and their input-o…
Many modern datacenter applications involve large-scale computations composed of multiple data flows that need to be completed over a shared set of distributed resources. Such a computation completes when all of its flows complete. A useful abstraction for modeling such scenarios is a {\em coflow}, which is a collection of flows (e.g., tasks, packets, data transmissions) that all share the same performance goal. In this paper, we present the first approximation algorithms for scheduling coflows over general network topologies with the objective of minimizing total weighted completion time. We …
Modern data centers face new scheduling challenges in optimizing job-level performance objectives, where a significant challenge is the scheduling of highly parallel data flows with a common performance goal (e.g., the shuffle operations in MapReduce applications). Chowdhury and Stoica introduced the coflow abstraction to capture these parallel communication patterns, and Chowdhury et al. proposed effective heuristics to schedule coflows efficiently. In our previous paper, we considered the strongly NP-hard problem of minimizing the total weighted completion time of coflows with release dates,…
The coflow scheduling problem has emerged as a popular abstraction in the last few years to study data communication problems within a data center. In this basic framework, each coflow has a set of communication demands and the goal is to schedule many coflows in a manner that minimizes the total weighted completion time. A coflow is said to complete when all its communication needs are met. This problem has been extremely well studied for the case of complete bipartite graphs that model a data center with full bisection bandwidth and several approximation algorithms and effective heuristics h…
Applications in data-parallel computing typically consist of multiple stages. In each stage, a set of intermediate parallel data flows (Coflow) is produced and transferred between servers to enable starting of next stage. While there has been much research on scheduling isolated coflows, the dependency between coflows in multi-stage jobs has been largely ignored. In this paper, we consider scheduling coflows of multi-stage jobs represented by general DAGs (Directed Acyclic Graphs) in a shared data center network, so as to minimize the total weighted completion time of jobs. This problem is sig…
Technische Universität Berlin(柏林工业大学);University of Pittsburgh(匹兹堡大学);Arizona State University(亚利桑那州立大学);Khoury College of Computer Sciences, Northeastern University(东北大学科尔里计算机学院)
In data-parallel computing frameworks, intermediate parallel data is often produced at various stages which needs to be transferred among servers in the datacenter network (e.g. the shuffle phase in MapReduce). A stage often cannot start or be completed unless all the required data pieces from the preceding stage are received. \emph{Coflow} is a recently proposed networking abstraction to capture such communication patterns. We consider the problem of efficiently scheduling coflows with release dates in a shared datacenter network so as to minimize the total weighted completion time of coflows…
In this paper, we introduce a new online scheduling framework for minimizing total weighted completion time in a general setting. The framework is inspired by the work of Hall et al. [Mathematics of Operations Research, Vol 22(3):513-544, 1997] and Garg et al. [Proc. of Foundations of Software Technology and Theoretical Computer Science, pp. 96-107, 2007], who show how to convert an offline approximation to an online scheme. Our framework uses two offline approximation algorithms (one for the simpler problem of scheduling without release times, and another for the minimum unscheduled weight pr…
In this paper, we study the peak-aware energy scheduling problem using the competitive framework with machine learning prediction. With the uncertainty of energy demand as the fundamental challenge, the goal is to schedule the energy output of local generation units such that the electricity bill is minimized. While this problem has been tackled using classic competitive design with worst-case guarantee, the goal of this paper is to develop learning-assisted competitive algorithms to improve the performance in a provable manner. We develop two deterministic and randomized algorithms that are p…
Designing and analyzing algorithms with provable performance guarantees enables efficient optimization problem solving in different application domains, e.g.\ communication networks, transportation, economics, and manufacturing. Despite the significant contributions of approximation algorithms in engineering, only limited and isolated works contribute from this perspective in process systems engineering. The current paper discusses three representative, NP-hard problems in process systems engineering: (i) pooling, (ii) process scheduling, and (iii) heat exchanger network synthesis. We survey r…