This paper considers online optimization of a renewal-reward system. A controller performs a sequence of tasks back-to-back. Each task has a random vector of parameters, called the task type vector, that affects the task processing options and also affects the resulting reward and time duration of the task. The probability distribution for the task type vector is unknown and the controller must learn to make efficient decisions so that time average reward converges to optimality. Prior work on such renewal optimization problems leaves open the question of optimal convergence time. This paper develops an algorithm with an optimality gap that decays like $O(1/\sqrt{k})$, where $k$ is the number of tasks processed. The same algorithm is shown to have faster $O(\log(k)/k)$ performance when the system satisfies a strong concavity property. The proposed algorithm uses an auxiliary variable that is updated according to a classic Robbins-Monro iteration. It makes online scheduling decisions at the start of each renewal frame based on this variable and on the observed task type. A matching converse is obtained for the strongly concave case by constructing an example system for which all algorithms have performance at best $Ω(\log(k)/k)$. A matching $Ω(1/\sqrt{k})$ converse is also shown for the general case without strong concavity.
This document is the third sub-report from the research project SATT (Samplanering av trafikpåverkande åtgärder och trafikflöden, modellstudie / Coordinated planning of temporary capacity restrictions and traffic flows, model study). The report summarizes the modelling alternatives that have been considered. The main result of the model study is that we propose a bi-level approach for handling the two parts of the problem, i.e. the scheduling of renewal projects and the subsequent adjustments of railway traffic flow. The contributions are: (1) a mixed integer formulation for scheduling project…
This paper considers an opportunistic scheduling problem over a renewal system. A controller observes a random event at the beginning of each renewal frame and then chooses an action in response to the event, which affects the duration of the frame, the amount of resources used, and a penalty metric. The goal is to make frame-wise decisions so as to minimize the time average penalty subject to time average resource constraints. This problem has applications to task processing and communication in data networks, as well as to certain classes of Markov decision problems. We formulate the problem…
In this paper, we study optimal control problems for multiclass GI/M/n+M queues in an alternating renewal (up-down) random environment in the Halfin-Whitt regime. Assuming that the downtimes are asymptotically negligible and only the service processes are affected, we show that the limits of the diffusion-scaled state processes under non-anticipative, preemptive, work-conserving scheduling policies, are controlled jump diffusions driven by a compound Poisson jump process. We establish the asymptotic optimality of the infinite-horizon discounted and long-run average (ergodic) problems for the q…
We consider the problem of optimizing time averages in systems with independent and identically distributed behavior over renewal frames. This includes scheduling and task processing to maximize utility in stochastic networks with variable length scheduling modes. Every frame, a new policy is implemented that affects the frame size and that creates a vector of attributes. An algorithm is developed for choosing policies on each frame in order to maximize a concave function of the time average attribute vector, subject to additional time average constraints. The algorithm is based on Lyapunov op…
We revisit the problem of maximising the expected length of increasing subsequence that can be selected from a marked Poisson process by an online strategy. Resorting to a natural size variable, the problem is represented in terms of a controlled partially deterministic Markov process with decreasing paths. Refining known estimates we obtain fairly complete asymptotic expansions for the moments, and using a renewal approximation give a novel proof of the central limit theorem for the length of selected subsequence under the optimal strategy.
In this study, we investigated several online and semi-online scheduling problems on two hierarchical machines with a common due date to maximize the total early work. For the pure online case, we designed an optimal online algorithm with a competitive ratio of $\sqrt 2$. For the case when the total processing time is known, we proposed an optimal semi-online algorithm with a competitive ratio of $\frac{4}{3}$. Additionally, for the cases when the largest processing time is known, we gave optimal algorithms with a competitive ratio of $\frac{6}{5}$ if the largest job is a lower hierarchy one, …
Online Scheduling for LLM Inference with KV Cache Constraints
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology（麻省理工学院电子工程与计算机科学系）；HKUST（香港科技大学）；Microsoft Research（微软研究院）；Sloan School of Management, Massachusetts Institute of Technology（斯隆管理学院，麻省理工学院）；Operations Research Center, Massachusetts Institute of Technology（运营研究中心，麻省理工学院）
We study web and mobile applications that are used to schedule advance service, from medical appointments to restaurant reservations. We model them as online weighted bipartite matching problems with non-stationary arrivals. We propose new algorithms with performance guarantees for this class of problems. Specifically, we show that the expected performance of our algorithms is bounded below by $1-\sqrt{\frac{2}π}\frac{1}{\sqrt{k}}+O(\frac{1}{k})$ times that of an optimal offline algorithm, which knows all future information upfront, where $k$ is the minimum capacity of a resource. This is the …
Demand-response operation of air separation units requires frequent changes in production rate(s), and scheduling calculations must explicitly consider process dynamics to ensure feasibility of the solutions. To this end, scale-bridging models (SBMs) approximate the scheduling-relevant dynamics of a process and its controller in a low-order representation. In contrast to previous works that have employed nonlinear SBMs, this paper proposes linear SBMs, developed using time-series analysis, to facilitate online scheduling computations. Using a year-long industrial dataset, we find that compact …
In this work, the online printing shop scheduling problem is considered. This challenging real problem, that appears in the nowadays printing industry, can be seen as a flexible job shop scheduling problem with sequence flexibility in which precedence constraints among operations of a job are given by an arbitrary directed acyclic graph. In addition, several complicating particularities such as periods of unavailability of the machines, resumable operations, sequence-dependent setup times, partial overlapping among operations with precedence constraints, release times, and fixed operations are…
Age of information is a new network performance metric that captures the freshness of information at end-users. This paper studies the age of information from a scheduling perspective. To that end, we consider a wireless broadcast network where a base-station (BS) is updating many users on random information arrivals under a transmission capacity constraint. For the offline case when the arrival statistics are known to the BS, we develop a structural MDP scheduling algorithm and an index scheduling algorithm, leveraging Markov decision process (MDP) techniques and the Whittle's methodology for…
This paper considers a cost minimization problem for data centers with N servers and randomly arriving service requests. A central router decides which server to use for each new request. Each server has three types of states (active, idle, setup) with different costs and time durations. The servers operate asynchronously over their own states and can choose one of multiple sleep modes when idle. We develop an online distributed control algorithm so that each server makes its own decisions, the request queues are bounded and the overall time average cost is near optimal with probability 1. The…
Polling systems have been widely studied, however most of these studies focus on polling systems with renewal processes for arrivals and random variables for service times. There is a need driven by practical applications to study polling systems with arbitrary arrivals (not restricted to time-varying or in batches) and revealed service time upon a job's arrival. To address that need, our work considers a polling system with generic setting and for the first time provides the worst-case analysis for online scheduling policies in this system. We provide conditions for the existence of constant …
We consider online preemptive scheduling of jobs arriving one by one, to be assigned to two identical machines, with the goal of makespan minimization. We study the effect of selecting the best solution out of two independent solutions constructed in parallel in an online fashion. Two cases are analyzed, where one case is purely online, and in the other one jobs are presented sorted by non-increasing sizes. We show that using two solutions rather than one improves the performance significantly, but that an optimal solution cannot be obtained for any constant number of solutions constructed in …
Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints
Institute for Data, Systems, and Society, Massachusetts Institute of Technology（数据、系统与社会研究所，麻省理工学院）；School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Alibaba Group（阿里巴巴集团）
