Coflow is a recently proposed network abstraction for data-parallel computing applications. This paper considers scheduling coflows with precedence constraints in identical parallel networks, such as to minimize the total weighted completion time of coflows. The identical parallel network is an architecture based on multiple network cores running in parallel. In the divisible coflow scheduling problem, the proposed algorithm achieves $(6-\frac{2}{m})μ$ and $(5-\frac{2}{m})μ$ approximate ratios for arbitrary release time and zero release time, respectively, where $m$ is the number of network cores and $μ$ is the coflow number of the longest path in the precedence graph. In the indivisible coflow scheduling problem, the proposed algorithm achieves $(4m+1)μ$ and $4mμ$ approximate ratios for arbitrary release time and zero release time, respectively. In the single network core scheduling problem, we propose a $5μ$-approximation algorithm with arbitrary release times, and a $4μ$-approximation without release time. Moreover, the proposed algorithm can be modified to solve the coflows of multi-stage jobs scheduling problem. In multi-stage jobs, coflow is transferred between servers to enable starting of next stage. This means that there are precedence constraints between coflows of job. Our result represents an improvement upon the previous best approximation ratio of $O(\tildeμ \log(N)/ \log(\log(N)))$ where $\tildeμ$ is the maximum number of coflows in a job and $N$ is the number of servers.
Efficient Approximation Algorithms for Scheduling Coflows with Precedence Constraints in Identical Parallel Networks to Minimize Weighted Completion Time
Coflow is a recently proposed network abstraction to capture communication patterns in data centers. The coflow scheduling problem in large data centers is one of the most important $NP$-hard problems. Previous research on coflow scheduling focused mainly on the single-switch model. However, with recent technological developments, this single-core model is no longer sufficient. This paper considers the coflow scheduling problem in identical parallel networks. The identical parallel network is an architecture based on multiple network cores running in parallel. Coflow can be considered as divis…
Coflow scheduling models communication requests in parallel computing frameworks where multiple data flows between shared resources need to be completed before computation can continue. In this paper, we introduce Path-based Coflow Scheduling, a generalized problem variant that considers coflows as collections of flows along fixed paths on general network topologies with node capacity restrictions. For this problem, we minimize the coflows' total weighted completion time. We show that flows on paths in the original network can be interpreted as hyperedges in a hypergraph and transform the path…
The coflow scheduling problem is considered: given an input/output switch with each port having a fixed capacity, find a scheduling algorithm that minimizes the weighted sum of the coflow completion times respecting the port capacities, where each flow of a coflow has a demand per input/output port, and coflow completion time is the finishing time of the last flow of the coflow. The objective of this paper is to present theoretical guarantees on approximating the sum of coflow completion time in the non-clairvoyant setting, where on a coflow arrival, only the number of flows, and their input-o…
Many modern datacenter applications involve large-scale computations composed of multiple data flows that need to be completed over a shared set of distributed resources. Such a computation completes when all of its flows complete. A useful abstraction for modeling such scenarios is a {\em coflow}, which is a collection of flows (e.g., tasks, packets, data transmissions) that all share the same performance goal. In this paper, we present the first approximation algorithms for scheduling coflows over general network topologies with the objective of minimizing total weighted completion time. We …
Modern data centers face new scheduling challenges in optimizing job-level performance objectives, where a significant challenge is the scheduling of highly parallel data flows with a common performance goal (e.g., the shuffle operations in MapReduce applications). Chowdhury and Stoica introduced the coflow abstraction to capture these parallel communication patterns, and Chowdhury et al. proposed effective heuristics to schedule coflows efficiently. In our previous paper, we considered the strongly NP-hard problem of minimizing the total weighted completion time of coflows with release dates,…
The coflow scheduling problem has emerged as a popular abstraction in the last few years to study data communication problems within a data center. In this basic framework, each coflow has a set of communication demands and the goal is to schedule many coflows in a manner that minimizes the total weighted completion time. A coflow is said to complete when all its communication needs are met. This problem has been extremely well studied for the case of complete bipartite graphs that model a data center with full bisection bandwidth and several approximation algorithms and effective heuristics h…
Applications in data-parallel computing typically consist of multiple stages. In each stage, a set of intermediate parallel data flows (Coflow) is produced and transferred between servers to enable starting of next stage. While there has been much research on scheduling isolated coflows, the dependency between coflows in multi-stage jobs has been largely ignored. In this paper, we consider scheduling coflows of multi-stage jobs represented by general DAGs (Directed Acyclic Graphs) in a shared data center network, so as to minimize the total weighted completion time of jobs. This problem is sig…
This paper presents different methods for solving parallel machine scheduling problems with precedence constraints and setup times between the jobs. Limited discrepancy search methods mixed with local search principles, dominance conditions and specific lower bounds are proposed. The proposed methods are evaluated on a set of randomly generated instances and compared with previous results from the literature and those obtained with an efficient commercial solver. We conclude that our propositions are quite competitive and our results even outperform other approaches in most cases.
Technische Universität Berlin（柏林工业大学）；University of Pittsburgh（匹兹堡大学）；Arizona State University（亚利桑那州立大学）；Khoury College of Computer Sciences, Northeastern University（东北大学科尔里计算机学院）
Scheduling a set of jobs over a collection of machines is a fundamental problem that needs to be solved millions of times a day in various computing platforms: in operating systems, in large data clusters, and in data centers. Along with makespan, flow-time, which measures the length of time a job spends in a system before it completes, is arguably the most important metric to measure the performance of a scheduling algorithm. In recent years, there has been a remarkable progress in understanding flow-time based objective functions in diverse settings such as unrelated machines scheduling, bro…
We consider the classic problem of scheduling a set of n jobs non-preemptively on a single machine. Each job j has non-negative processing time, weight, and deadline, and a feasible schedule needs to be consistent with chain-like precedence constraints. The goal is to compute a feasible schedule that minimizes the sum of penalties of late jobs. Lenstra and Rinnoy Kan [Annals of Disc. Math., 1977] in their seminal work introduced this problem and showed that it is strongly NP-hard, even when all processing times and weights are 1. We study the approximability of the problem and our main result …
A central problem in scheduling is to schedule $n$ unit size jobs with precedence constraints on $m$ identical machines so as to minimize the makespan. For $m=3$, it is not even known if the problem is NP-hard and this is one of the last open problems from the book of Garey and Johnson. We show that for fixed $m$ and $ε$, $(\log n)^{O(1)}$ rounds of Sherali-Adams hierarchy applied to a natural LP of the problem provides a $(1+ε)$-approximation algorithm running in quasi-polynomial time. This improves over the recent result of Levey and Rothvoss, who used $r=(\log n)^{O(\log \log n)}$ rounds of…
In a classical scheduling problem, we are given a set of $n$ jobs of unit length along with precedence constraints and the goal is to find a schedule of these jobs on $m$ identical machines that minimizes the makespan. This problem is well-known to be NP-hard for an unbounded number of machines. Using standard 3-field notation, it is known as $P|\text{prec}, p_j=1|C_{\max}$. We present an algorithm for this problem that runs in $O(1.995^n)$ time. Before our work, even for $m=3$ machines the best known algorithms ran in $O^\ast(2^n)$ time. In contrast, our algorithm works when the number of mac…
Scheduling jobs with precedence constraints on a set of identical machines to minimize the total processing time (makespan) is a fundamental problem in combinatorial optimization. In practical settings such as cloud computing, jobs are often malleable, i.e., can be processed on multiple machines simultaneously. The instantaneous processing rate of a job is a non-decreasing function of the number of machines assigned to it (we call it the processing function). Previous research has focused on practically relevant concave processing functions, which obey the law of diminishing utility and genera…
We consider non-preemptive scheduling of MapReduce jobs with multiple tasks in the practical scenario where each job requires several map-reduce rounds. We seek to minimize the average weighted completion time and consider scheduling on identical and unrelated parallel processors. For identical processors, we present LP-based O(1)-approximation algorithms. For unrelated processors, the approximation ratio naturally depends on the maximum number of rounds of any job. Since the number of rounds per job in typical MapReduce algorithms is a small constant, our scheduling algorithms achieve a small…
We consider the problem of scheduling a set of $n$ tasks on $m$ processors under precedence, communication, and global system energy constraints to minimize makespan. We extend existing scheduling models to account for energy usage and give convex programming algorithms that yield essentially the same results as existing algorithms that do not consider energy, while adhering to a strict energy bound.
We give a $(2 + ε)$-approximation algorithm for minimizing total weighted completion time on a single machine under release time and precedence constraints. This settles a recent conjecture made in [18]
We study the classical scheduling problem on parallel machines %with precedence constraints where the precedence graph has the bounded depth $h$. Our goal is to minimize the maximum completion time. We focus on developing approximation algorithms that use only sublinear space or sublinear time. We develop the first one-pass streaming approximation schemes using sublinear space when all jobs' processing times differ no more than a constant factor $c$ and the number of machines $m$ is at most $\tfrac {2n ε}{3 h c }$. This is so far the best approximation we can have in terms of $m$, since no pol…
In the scheduling with non-uniform communication delay problem, the input is a set of jobs with precedence constraints. Associated with every precedence constraint between a pair of jobs is a communication delay, the time duration the scheduler has to wait between the two jobs if they are scheduled on different machines. The objective is to assign the jobs to machines to minimize the makespan of the schedule. Despite being a fundamental problem in theory and a consequential problem in practice, the approximability of scheduling problems with communication delays is not very well understood. On…
We study the classic problem of scheduling $n$ precedence constrained unit-size jobs on $m = O(1)$ machines so as to minimize the makespan. In a recent breakthrough, Levey and Rothvoss \cite{LR16} developed a $(1+ε)$-approximation for the problem with running time $\exp\Big(\exp\Big(O\big(\frac{m^2}{ε^2}\log^2\log n\big)\Big)\Big)$, via the Sherali-Adams lift of the basic linear programming relaxation for the problem by $\exp\Big(O\big(\frac{m^2}{ε^2}\log^2\log n\big)\Big)$ levels. Garg \cite{Garg18} recently improved the number of levels to $\log ^{O(m^2/ε^2)}n$, and thus the running time to …
In data-parallel computing frameworks, intermediate parallel data is often produced at various stages which needs to be transferred among servers in the datacenter network (e.g. the shuffle phase in MapReduce). A stage often cannot start or be completed unless all the required data pieces from the preceding stage are received. \emph{Coflow} is a recently proposed networking abstraction to capture such communication patterns. We consider the problem of efficiently scheduling coflows with release dates in a shared datacenter network so as to minimize the total weighted completion time of coflows…
We consider the problem of efficiently scheduling jobs with precedence constraints on a set of identical machines in the presence of a uniform communication delay. Such precedence-constrained jobs can be modeled as a directed acyclic graph, $G = (V, E)$. In this setting, if two precedence-constrained jobs $u$ and $v$, with $v$ dependent on $u$ ($u \prec v$), are scheduled on different machines, then $v$ must start at least $ρ$ time units after $u$ completes. The scheduling objective is to minimize makespan, i.e. the total time from when the first job starts to when the last job finishes. The f…
We consider the classic problem of scheduling jobs with precedence constraints on a set of identical machines to minimize the makespan objective function. Understanding the exact approximability of the problem when the number of machines is a constant is a well-known question in scheduling theory. Indeed, an outstanding open problem from the classic book of Garey and Johnson asks whether this problem is NP-hard even in the case of 3 machines and unit-length jobs. In a recent breakthrough, Levey and Rothvoss gave a $(1+ε)$-approximation algorithm, which runs in nearly quasi-polynomial time, for…
We study a natural variant of scheduling that we call \emph{partial scheduling}: In this variant an instance of a scheduling problem along with an integer $k$ is given and one seeks an optimal schedule where not all, but only $k$ jobs, have to be processed. Specifically, we aim to determine the fine-grained parameterized complexity of partial scheduling problems parameterized by $k$ for all variants of scheduling problems that minimize the makespan and involve unit/arbitrary processing times, identical/unrelated parallel machines, release/due dates, and precedence constraints. That is, we inve…
The scheduling literature has traditionally focused on a single type of resource (e.g., computing nodes). However, scientific applications in modern High-Performance Computing (HPC) systems process large amounts of data, hence have diverse requirements on different types of resources (e.g., cores, cache, memory, I/O). All of these resources could potentially be exploited by the runtime scheduler to improve the application performance. In this paper, we study multi-resource scheduling to minimize the makespan of computational workflows comprised of parallel jobs subject to precedence constraint…
Distributed computing systems often need to consider the scheduling problem involving a collection of highly dependent data-processing tasks that must work in concert to achieve mission-critical objectives. This paper considers the unrelated machine scheduling problem for minimizing weighted sum completion time under arbitrary precedence constraints and on heterogeneous machines with different processing speeds. The problem is known to be strongly NP-hard even in the single machine setting. By making use of Queyranne's constraint set and constructing a novel Linear Programming relaxation for t…
In the SCHED problem we are given a set of n jobs, together with their processing times and precedence constraints. The task is to order the jobs so that their total completion time is minimized. SCHED is a special case of the Traveling Repairman Problem with precedences. A natural dynamic programming algorithm solves both these problems in 2^n n^O(1) time, and whether there exists an algorithms solving SCHED in O(c^n) time for some constant c < 2 was an open problem posted in 2004 by Woeginger. In this paper we answer this question positively.
We consider the online problem of scheduling jobs on identical machines, where jobs have precedence constraints. We are interested in the demanding setting where the jobs sizes are not known up-front, but are revealed only upon completion (the non-clairvoyant setting). Such precedence-constrained scheduling problems routinely arise in map-reduce and large-scale optimization. In this paper, we make progress on this problem. For the objective of total weighted completion time, we give a constant-competitive algorithm. And for total weighted flow-time, we give an $O(1/ε^2)$-competitive algorithm …
In a classical problem in scheduling, one has $n$ unit size jobs with a precedence order and the goal is to find a schedule of those jobs on $m$ identical machines as to minimize the makespan. It is one of the remaining four open problems from the book of Garey & Johnson whether or not this problem is $\mathbf{NP}$-hard for $m=3$. We prove that for any fixed $\varepsilon$ and $m$, an LP-hierarchy lift of the time-indexed LP with a slightly super poly-logarithmic number of $r = (\log(n))^{Θ(\log \log n)}$ rounds provides a $(1 + \varepsilon)$-approximation. For example Sherali-Adams suffices as…
Inapproximability of Unique-Machine Precedence Scheduling for Unit-Length Jobs
Simons Institute for the Theory of Computing, and Departments of EECS & Mathematics, UC Berkeley（加州大学伯克利分校）；Department of EECS, UC Berkeley（加州大学伯克利分校）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）
Resource Constrained Project Scheduling Problems (RCPSPs) without preemption are well-known NP-hard combinatorial optimization problems. A feasible RCPSP solution consists of a time-ordered schedule of jobs with corresponding execution modes, respecting precedence and resources constraints. In this paper, we propose a cutting plane algorithm to separate five different cut families, as well as a new preprocessing routine to strengthen resource-related constraints. New lifted versions of the well-known precedence and cover inequalities are employed. At each iteration, a dense conflict graph is b…
This paper presents improved approximation algorithms for the problem of multiprocessor scheduling under uncertainty, or SUU, in which the execution of each job may fail probabilistically. This problem is motivated by the increasing use of distributed computing to handle large, computationally intensive tasks. In the SUU problem we are given n unit-length jobs and m machines, a directed acyclic graph G of precedence constraints among jobs, and unrelated failure probabilities q_{ij} for each job j when executed on machine i for a single timestep. Our goal is to find a schedule that minimizes th…
We consider the problem of scheduling $n$ precedence-constrained jobs on $m$ uniformly-related machines in the presence of an arbitrary, fixed communication delay $ρ$. We consider a model that allows job duplication, i.e. processing of the same job on multiple machines, which, as we show, can reduce the length of a schedule (i.e., its makespan) by a logarithmic factor. Our main result is an $O(\log m \log ρ/ \log \log ρ)$-approximation algorithm for minimizing makespan, assuming the minimum makespan is at least $ρ$. Our algorithm is based on rounding a linear programming relaxation for the pro…
We consider online scheduling to minimize weighted completion time on related machines, where each job consists of several tasks that can be concurrently executed. A job gets completed when all its component tasks finish. We obtain an $O(K^3 \log^2 K)$-competitive algorithm in the non-clairvoyant setting, where $K$ denotes the number of distinct machine speeds. The analysis is based on dual-fitting on a precedence-constrained LP relaxation that may be of independent interest.
We study the problem of scheduling precedence-constrained jobs on heterogenous machines in the presence of non-uniform job and machine communication delays. We are given as input $n$ unit size precedence-ordered jobs and $m$ related machines such that machine $i$ can execute up to $m_i$ jobs at a time. Each machine $i$ has an in-delay $ρ^{\mathrm{in}}_i$ and out-delay $ρ^{\mathrm{out}}_i$. Likewise, each job $v$ has an in-delay $ρ^{\mathrm{in}}_v$ and out-delay $ρ^{\mathrm{out}}_v$. In a schedule, job $v$ may be executed on machine $i$ at time $t$ if each predecessor $u$ of $v$ is completed on…
We consider single-machine scheduling problems that are natural generalizations or variations of the min-sum set cover problem and the min-sum vertex cover problem. For each of these problems, we give new approximation algorithms. Some of these algorithms rely on time-indexed LP relaxations. We show how a variant of alpha-point scheduling leads to the best-known approximation ratios, including a guarantee of 4 for an interesting special case of the so-called generalized min-sum set cover problem. We also make explicit the connection between the greedy algorithm for min-sum set cover and the co…
In this paper, we introduce a new online scheduling framework for minimizing total weighted completion time in a general setting. The framework is inspired by the work of Hall et al. [Mathematics of Operations Research, Vol 22(3):513-544, 1997] and Garg et al. [Proc. of Foundations of Software Technology and Theoretical Computer Science, pp. 96-107, 2007], who show how to convert an offline approximation to an online scheme. Our framework uses two offline approximation algorithms (one for the simpler problem of scheduling without release times, and another for the minimum unscheduled weight pr…
We consider non-clairvoyant scheduling with online precedence constraints, where an algorithm is oblivious to any job dependencies and learns about a job only if all of its predecessors have been completed. Given strong impossibility results in classical competitive analysis, we investigate the problem in a learning-augmented setting, where an algorithm has access to predictions without any quality guarantee. We discuss different prediction models: novel problem-specific models as well as general ones, which have been proposed in previous works. We present lower bounds and algorithmic upper bo…
Parameterized Complexity of Scheduling Problems in Robotic Process Automation
Czech Technical University in Prague（布拉格捷克技术大学）；Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague（捷克技术大学捷克斯洛伐克信息学、机器人学与自动化研究所）；Sorbonne University（索邦大学）；CNRS（法国国家科学研究中心）
We study approximation algorithms for scheduling problems with the objective of minimizing total weighted completion time, under identical and related machine models with job precedence constraints. We give algorithms that improve upon many previous 15 to 20-year-old state-of-art results. A major theme in these results is the use of time-indexed linear programming relaxations. These are natural relaxations for their respective problems, but surprisingly are not studied in the literature. We also consider the scheduling problem of minimizing total weighted completion time on unrelated machines.…
Modern platforms are using accelerators in conjunction with standard processing units in order to reduce the running time of specific operations, such as matrix operations, and improve their performance. Scheduling on such hybrid platforms is a challenging problem since the algorithms used for the case of homogeneous resources do not adapt well. In this paper we consider the problem of scheduling a set of tasks subject to precedence constraints on hybrid platforms, composed of two types of processing units. We propose a $(3+2\sqrt{2})$-approximation algorithm and a conditional lower bound of 3…
A multiple server setting is considered, where each server has tunable speed, and increasing the speed incurs an energy cost. Jobs arrive to a single queue, and each job has two types of sub-tasks, map and reduce, and a {\bf precedence} constraint among them: any reduce task of a job can only be processed once all the map tasks of the job have been completed. In addition to the scheduling problem, i.e., which task to execute on which server, with tunable speed, an additional decision variable is the choice of speed for each server, so as to minimize a linear combination of the sum of the flow …
The paper presents complexity results and performance guaranties for a family of approximation algorithms for an optimisation problem arising in software testing and manufacturing. The problem is formulated as a partitioning of a set where each element has an associated subset in another set, but can also be viewed as a scheduling problem with infinitely large communication delay, precedence constraints in the form of a bipartite graph, and duplication.
