In this paper, we investigate the combination of synthesis, model-based learning, and online sampling techniques to obtain safe and near-optimal schedulers for a preemptible task scheduling problem. Our algorithms can handle Markov decision processes (MDPs) that have 1020 states and beyond which cannot be handled with state-of-the art probabilistic model-checkers. We provide probably approximately correct (PAC) guarantees for learning the model. Additionally, we extend Monte-Carlo tree search with advice, computed using safety games or obtained using the earliest-deadline-first scheduler, to safely explore the learned model online. Finally, we implemented and compared our algorithms empirically against shielded deep Q-learning on large task systems.
This paper proposes a safe reinforcement learning algorithm for generation bidding decisions and unit maintenance scheduling in a competitive electricity market environment. In this problem, each unit aims to find a bidding strategy that maximizes its revenue while concurrently retaining its reliability by scheduling preventive maintenance. The maintenance scheduling provides some safety constraints which should be satisfied at all times. Satisfying the critical safety and reliability constraints while the generation units have an incomplete information of each others' bidding strategy is a ch…
Safe Hierarchical Reinforcement Learning for CubeSat Task Scheduling Based on Energy Consumption
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg (UL)（卢森堡大学跨学科安全、可靠性与信任中心（SnT））；Automation and Robotics Research Group (ARG)（自动化与机器人研究组）；Space Systems Research Group (SpaSys)（空间系统研究组）
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NASA Langley Research Center（美国国家航空航天局兰利研究中心）；Skygrid LLC（Skygrid有限责任公司）
This work considers the scheduling problem for Urban Air Mobility (UAM) vehicles travelling between origin-destination pairs with both hard and soft trip deadlines. Each route is described by a discrete probability distribution over trip completion times (or delay) and over inter-arrival times of requests (or demand) for the route along with a fixed hard or soft deadline. Soft deadlines carry a cost that is incurred when the deadline is missed. An online, safe scheduler is developed that ensures that hard deadlines are never missed, and that average cost of missing soft deadlines is minimized.…
Agentic-V2X: Small Language Model Agents for Deadline-Aware V2X Scheduling in 5G/6G Networks
Department of Informatics, University of Piraeus（比雷埃克斯大学信息学院）；Department of Informatics and Telecommunications, University of Peloponnese（希腊佩拉索斯大学信息与电信学院）
The Pump Scheduling Problem: A Real-World Scenario for Reinforcement Learning
Normandie Université（诺曼底大学）；INSA Rouen（里昂大学鲁昂分校）；LITIS Rouen（鲁昂LITIS研究所）；Technical University of Kaiserslautern（凯撒斯劳滕技术大学）；SAM Kaiserlaustern（凯撒斯劳滕SAM）
Beijing University of Posts and Telecommunications（北京邮电大学）；Massachusetts Institute of Technology（麻省理工学院）；IFLYTEK Co., Ltd（科大讯飞股份有限公司）
The ever-increasing heavy traffic congestion potentially impedes the accessibility of emergency vehicles (EVs), resulting in detrimental impacts on critical services and even safety of people's lives. Hence, it is significant to propose an efficient scheduling approach to help EVs arrive faster. Existing vehicle-centric scheduling approaches aim to recommend the optimal paths for EVs based on the current traffic status while the road-centric scheduling approaches aim to improve the traffic condition and assign a higher priority for EVs to pass an intersection. With the intuition that real-time…
Byju’s Lab（Byju实验室）；Technische Universität Darmstadt（达姆施塔特工业大学）；University College London（伦敦大学学院）；Huawei R&D（华为研发中心）
Safe exploration is a challenging and important problem in model-free reinforcement learning (RL). Often the safety cost is sparse and unknown, which unavoidably leads to constraint violations -- a phenomenon ideally to be avoided in safety-critical applications. We tackle this problem by augmenting the state-space with a safety state, which is nonnegative if and only if the constraint is satisfied. The value of this state also serves as a distance toward constraint violation, while its initial value indicates the available safety budget. This idea allows us to derive policies for scheduling t…
Tsinghua-Berkeley Shenzhen Institute, Tsinghua University（清华大学清华-伯克利深圳学院）
This paper addresses the problem of optimizing charging/discharging schedules of electric vehicles (EVs) when participate in demand response (DR). As there exist uncertainties in EVs' remaining energy, arrival and departure time, and future electricity prices, it is quite difficult to make charging decisions to minimize charging cost while guarantee that the EV's battery state-of-the-charge (SOC) is within certain range. To handle with this dilemma, this paper formulates the EV charging scheduling problem as a constrained Markov decision process (CMDP). By synergistically combining the augment…
Research in UAV scheduling has obtained an emerging interest from scientists in the optimization field. When the scheduling itself has established a strong root since the 19th century, works on UAV scheduling in indoor environment has come forth in the latest decade. Several works on scheduling UAV operations in indoor (two and three dimensional) and outdoor environments are reported. In this paper, a further study on UAV scheduling in three dimensional indoor environment is investigated. Dealing with indoor environment\textemdash where humans, UAVs, and other elements or infrastructures are l…
Towards Secure and Efficient Data Scheduling for Vehicular Social Networks
Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；College of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；School of Science, Computing and Engineering Technologies, Swinburne University of Technology（斯威本科技大学科学、计算与工程技术学院）
NICTA（澳大利亚国家信息与通信技术研究院）；University of Melbourne（墨尔本大学）；Australian National University（澳大利亚国立大学）
Large-scale controlled evacuations require emergency services to select evacuation routes, decide departure times, and mobilize resources to issue orders, all under strict time constraints. Existing algorithms almost always allow for preemptive evacuation schedules, which are less desirable in practice. This paper proposes, for the first time, a constraint-based scheduling model that optimizes the evacuation flow rate (number of vehicles sent at regular time intervals) and evacuation phasing of widely populated areas, while ensuring a nonpreemptive evacuation for each residential zone. Two opt…
Safely serving the school transportation demand with the minimum number of buses is one of the highest financial goals of school transportation directors. To achieve that objective, a good and efficient way to solve the routing and scheduling problem is required. Due to the growth of the computing power, the spotlight has been shed on solving the combined problem of the school bus routing and scheduling problem. We show that an integrated multi-school bus routing and scheduling can be formulated with the help of trip compatibility. A novel decomposition algorithm is proposed to solve the integ…
University of Zagreb（萨格勒布大学）；Universidad de Sevilla（塞维利亚大学）
To enable safe and efficient use of multi-robot systems in everyday life, a robust and fast method for coordinating their actions must be developed. In this paper, we present a distributed task allocation and scheduling algorithm for missions where the tasks of different robots are tightly coupled with temporal and precedence constraints. The approach is based on representing the problem as a variant of the vehicle routing problem, and the solution is found using a distributed metaheuristic algorithm based on evolutionary computation (CBM-pop). Such an approach allows a fast and near-optimal a…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Microsoft Research（微软研究院）；Cornell University（康奈尔大学）
We consider a scheduling problem where a cloud service provider has multiple units of a resource available over time. Selfish clients submit jobs, each with an arrival time, deadline, length, and value. The service provider's goal is to implement a truthful online mechanism for scheduling jobs so as to maximize the social welfare of the schedule. Recent work shows that under a stochastic assumption on job arrivals, there is a single-parameter family of mechanisms that achieves near-optimal social welfare. We show that given any such family of near-optimal online mechanisms, there exists an onl…
