Age of information is a new concept that characterizes the freshness of information at end devices. This paper studies the age of information from a scheduling perspective. We consider a wireless broadcast network where a base-station updates many users on stochastic information arrivals. Suppose that only one user can be updated for each time. In this context, we aim at developing a transmission scheduling algorithm for minimizing the long-run average age. To develop a low-complexity transmission scheduling algorithm, we apply the Whittle's framework for restless bandits. We successfully derive the Whittle index in a closed form and establish the indexability. Based on the Whittle index, we propose a scheduling algorithm, while experimentally showing that it closely approximates an age-optimal scheduling algorithm.
Age of information is a new network performance metric that captures the freshness of information at end-users. This paper studies the age of information from a scheduling perspective. To that end, we consider a wireless broadcast network where a base-station (BS) is updating many users on random information arrivals under a transmission capacity constraint. For the offline case when the arrival statistics are known to the BS, we develop a structural MDP scheduling algorithm and an index scheduling algorithm, leveraging Markov decision process (MDP) techniques and the Whittle's methodology for…
The problem of stochastic deadline scheduling is considered. A constrained Markov decision process model is introduced in which jobs arrive randomly at a service center with stochastic job sizes, rewards, and completion deadlines. The service provider faces random processing costs, convex non-completion penalties, and a capacity constraint that limits the simultaneous processing of jobs. Formulated as a restless multi-armed bandit problem, the stochastic deadline scheduling problem is shown to be indexable. A closed-form expression of the Whittle's index is obtained for the case when the proce…
This paper proposes a dynamic primal-dual type algorithm to solve the optimal scheduling problem in wireless networks subject to uncertain parameters, which are generated by stochastic network processes such as random packet arrivals, channel fading, and node mobilities. The algorithm is a generalization of the well-known max-weight scheduling algorithm proposed by Tassiulas et al., where only queue length information is used for computing the schedules when the arrival rates are uncertain. Using the technique of fluid limits, sample path convergence of the algorithm to an arbitrarily close to…
In this paper we present a stable day-to-day dynamical system for drivers' departure time choice at a single bottleneck. We first define within-day traffic dynamics with the point queue model, costs, the departure time user equilibrium (DTUE), and the arrival time user equilibrium (ATUE). We then identify three behavioral principles: (i) Drivers choose their departure and arrival times in a backward fashion (backward choice principle); (ii) After choosing the arrival times, they update their departure times to balance the total costs (cost balancing principle); (iii) They choose their arrival …
The number of cancer cases per year is rapidly increasing worldwide. In radiation therapy (RT), radiation from linear accelerators is used to kill malignant tumor cells. Scheduling patients for RT is difficult both due to the numerous medical and technical constraints, and because of the stochastic inflow of patients with different urgency levels. In this paper, a Column Generation (CG) approach is proposed for the RT patient scheduling problem. The model includes all the constraints necessary for the generated schedules to work in practice, including for example different machine compatibilit…
Deep Reinforcement Learning for Flexible Job Shop Scheduling with Random Job Arrivals
Centre for Artificial Intelligence, Zurich University of Applied Sciences（人工智能中心，苏黎世应用科学大学）；Inspire AG（Inspire公司）；Automatic Control Laboratory, ETH Zurich（自动控制实验室，苏黎世联邦理工学院）
The cooperation of connected and automated vehicles (CAVs) has shown great potential in improving traffic efficiency during intersection management. Existing research mainly focuses on intersections where lane changing is prohibited, which is impractical for real-life implementation. This paper proposes a two-stage cooperation framework, which decouples the longitudinal and lateral control of CAVs, allowing them to change to their preferred lanes. Based on formation control, an iterative framework is initially proposed to solve the target assignment and path planning problem for multiple CAVs …
The successful launch of electric vehicles (EVs) depends critically on the availability of convenient and economic charging facilities. The problem of scheduling of large-scale charging of EVs by a service provider is considered. A Markov decision process model is introduced in which EVs arrive randomly at a charging facility with random demand and completion deadlines. The service provider faces random charging costs, convex non-completion penalties, and a peak power constraint that limits the maximum number of simultaneous activation of EV chargers. Formulated as a restless multi-armed bandi…
The Whittle index, which characterizes optimal policies for controlling certain single restless bandit projects (a Markov decision process with two actions: active and passive) is the basis for a widely used heuristic index policy for the intractable restless multiarmed bandit problem. Yet two roadblocks need to be overcome to apply such a policy: the individual projects in the model at hand must be shown to be indexable, so that they possess a Whittle index; and the index must be evaluated. Such roadblocks can be especially vexing when project state spaces are real intervals, as in recent sen…
In the paper, we consider delay-optimal charging scheduling of the electric vehicles (EVs) at a charging station with multiple charge points. The charging station is equipped with renewable energy generation devices and can also buy energy from power grid. The uncertainty of the EV arrival, the intermittence of the renewable energy, and the variation of the grid power price are taken into account and described as independent Markov processes. Meanwhile, the charging energy for each EV is random. The goal is to minimize the mean waiting time of EVs under the long term constraint on the cost. We…
The Whittle index, which characterizes optimal policies for controlling certain single restless bandit projects (a Markov decision process with two actions: active and passive) is the basis for a widely used heuristic index policy for the intractable restless multiarmed bandit problem. Yet two roadblocks need to be overcome to apply such a policy: the individual projects in the model at hand must be shown to be indexable, so that they possess a Whittle index; and the index must be evaluated. Such roadblocks can be especially vexing when project state spaces are real intervals, as in recent sen…
Tsinghua University（清华大学）；University of Southern California（南加州大学）
In this paper, we study scheduling of a queueing system with zero knowledge of instantaneous network conditions. We consider a one-hop single-server queueing system consisting of $K$ queues, each with time-varying and non-stationary arrival and service rates. Our scheduling approach builds on an innovative combination of adversarial bandit learning and Lyapunov drift minimization, without knowledge of the instantaneous network state (the arrival and service rates) of each queue. We then present two novel algorithms \texttt{SoftMW} (SoftMaxWeight) and \texttt{SSMW} (Sliding-window SoftMaxWeight…
The scheduling utility plays a fundamental role in addressing the commuting travel behaviours. In this paper, a new scheduling utility, termed as DMRD-SU, was suggested based on some recent research findings in behavioural economics. DMRD-SU admitted the existence of positive arrival-caused utility. In addition, besides the travel-time-caused utility and arrival-caused utility, DMRD-SU firstly took the departure utility into account. The necessity of the departure utility in trip scheduling was analysed comprehensively, and the corresponding individual trip scheduling model was presented. Base…
In this paper, we study the joint routing-scheduling problem in energy harvesting communication networks. Our policies, which are based on stochastic subgradient methods on the dual domain, act as an energy harvesting variant of the stochastic family of backpresure algorithms. Specifically, we propose two policies: (i) the Stochastic Backpressure with Energy Harvesting (SBP-EH), in which a node's routing-scheduling decisions are determined by the difference between the Lagrange multipliers associated to their queue stability constraints and their neighbors'; and (ii) the Stochastic Soft Backpr…
Polling systems have been widely studied, however most of these studies focus on polling systems with renewal processes for arrivals and random variables for service times. There is a need driven by practical applications to study polling systems with arbitrary arrivals (not restricted to time-varying or in batches) and revealed service time upon a job's arrival. To address that need, our work considers a polling system with generic setting and for the first time provides the worst-case analysis for online scheduling policies in this system. We provide conditions for the existence of constant …
We consider a stochastic, dynamic runway scheduling problem involving aircraft landings on a single runway. Sequencing decisions are made with knowledge of the estimated arrival times (ETAs) of all aircraft due to arrive at the airport, and these ETAs vary according to continuous-time stochastic processes. Time separations between consecutive runway landings are modeled via sequence-dependent Erlang distributions and are affected by weather conditions, which also evolve continuously over time. The resulting multi-stage optimization problem is intractable using exact methods and we propose a no…
We study web and mobile applications that are used to schedule advance service, from medical appointments to restaurant reservations. We model them as online weighted bipartite matching problems with non-stationary arrivals. We propose new algorithms with performance guarantees for this class of problems. Specifically, we show that the expected performance of our algorithms is bounded below by $1-\sqrt{\frac{2}π}\frac{1}{\sqrt{k}}+O(\frac{1}{k})$ times that of an optimal offline algorithm, which knows all future information upfront, where $k$ is the minimum capacity of a resource. This is the …
It is commonly seen that buses are blocked by the ones in front serving passengers and have to queue outside a curbside bus stop although there are vacant berths at the stop. The resultant bus delays degrade the service level of urban public transportation. A potential solution is to reschedule the arrivals of the buses at the stop for full utilization of the berths with the aid of connected vehicle technologies. This study proposes a mixed-integer linear programming model to optimize the scheduling of bus arrivals and the bus-berth matching at a curbside stop under connected vehicle environme…
In (Jin, 2018), a new day-to-day dynamical system was proposed for drivers' departure time choice at a single bottleneck. Based on three behavioral principles, the nonlocal departure and arrival times choice problems were converted to the local scheduling payoff choice problem, whose day-to-day dynamics are described by the Lighthill-Whitham-Richards (LWR) model on an imaginary road of increasing scheduling payoff. Thus the departure time user equilibrium (DTUE), the arrival time user equilibrium (ATUE), and the scheduling payoff user equilibrium (SPUE) are uniquely determined by the stationar…
We consider the problem of scheduling arrivals to a congestion system with a finite number of users having identical deterministic demand sizes. The congestion is of the processor sharing type in the sense that all users in the system at any given time are served simultaneously. However, in contrast to classical processor sharing congestion models, the processing slowdown is proportional to the number of users in the system at any time. That is, the rate of service experienced by all users is linearly decreasing with the number of users. For each user there is an ideal departure time (due date…
We consider the delay properties of max-weight opportunistic scheduling in a multi-user ON/OFF wireless system, such as a multi-user downlink or uplink. It is well known that max-weight scheduling stabilizes the network (and hence yields maximum throughput) whenever input rates are inside the network capacity region. We show that when arrival and channel processes are independent, average delay of the max-weight policy is order-optimal, in the sense that it does not grow with the number of network links. While recent queue-grouping algorithms are known to also yield order-optimal delay, this i…
