Single- vs. Dual-Policy Reinforcement Learning for Dynamic Bike Rebalancing
Department of Mathematical and Industrial Engineering, Polytechnique Montréal（蒙特利尔理工学院数学与工业工程系）；School of Management, Université du Québec à Montréal（魁北克大学蒙特利尔分校管理学院）；Cornell Tech and Technion - IIT（康奈尔科技与技术Ion研究所）
Bike-sharing systems (BSS) provide a sustainable urban mobility solution, but ensuring their reliability requires effective rebalancing strategies to address stochastic demand and prevent station imbalances. This paper proposes reinforcement learning (RL) algorithms for dynamic rebalancing problem with multiple vehicles, introducing and comparing two RL approaches: Single-policy RL and Dual-policy RL. We formulate this network optimization problem as a Markov Decision Process within a continuous-time framework, allowing vehicles to make independent and cooperative rebalancing decisions without synchronization constraints. In the first approach, a single deep Q-network (DQN) is trained to jointly learn inventory and routing decisions. The second approach decouples node-level inventory decisions from arc-level vehicle routing, enhancing learning efficiency and adaptability. A high-fidelity simulator under the first-arrive-first-serve rule is developed to estimate rewards across diverse demand scenarios influenced by temporal and weather variations. Extensive experiments demonstrate that while the single-policy model is competitive against several benchmarks, the dual-policy model significantly reduces lost demand. These findings provide valuable insights for bike-sharing operators, reinforcing the potential of RL for real-time rebalancing and paving the way for more adaptive and intelligent urban mobility solutions.
Dual Policy Reinforcement Learning for Real-time Rebalancing in Bike-sharing Systems
Polytechnique Montréal（蒙特利尔理工学院）；Université du Québec à Montréal（魁北克大学蒙特利尔分校）；Cornell Tech（康奈尔科技学院）；Technion – IIT（以色列理工学院）；Centre Interuniversitaire de Recherche sur les Reseaux d’Entreprise, la Logistique et le Transport (CIRRELT)（企业网络、物流与运输大学间研究中心）；SCALE-AI Chair in Data-Driven Supply Chains（SCALE-AI 数据驱动供应链讲席）；Canada Excellence Research Chair in Data-Science for Real-time Decision-Making (CERC)（加拿大实时决策数据科学卓越研究讲席）
SmartFlow Reinforcement Learning and Agentic AI for Bike-Sharing Optimisation
Department of Data Science, Machine Learning \& Artificial Intelligence, PES University, Bengaluru, Karnataka - 560100, India；1 Sunrise Setting Ltd, 12a Fore Street, St. Marychurch, Torquay, Devon, TQ1 4NE, UK 2 European Alliance for Innovation (EAI), Gent, Belgium
Offline reinforcement learning (RL) is challenged by the distributional shift between learning policies and datasets. To address this problem, existing works mainly focus on designing sophisticated algorithms to explicitly or implicitly constrain the learned policy to be close to the behavior policy. The constraint applies not only to well-performing actions but also to inferior ones, which limits the performance upper bound of the learned policy. Instead of aligning the densities of two distributions, aligning the supports gives a relaxed constraint while still being able to avoid out-of-dist…
University of South Florida（南佛罗里达大学）；KAIST（韩国科学技术院）
This paper proposes a reinforcement learning approach for nightly offline rebalancing operations in free-floating electric vehicle sharing systems (FFEVSS). Due to sparse demand in a network, FFEVSS require relocation of electrical vehicles (EVs) to charging stations and demander nodes, which is typically done by a group of drivers. A shuttle is used to pick up and drop off drivers throughout the network. The objective of this study is to solve the shuttle routing problem to finish the rebalancing work in the minimal time. We consider a reinforcement learning framework for the problem, in whic…
Fleet Rebalancing for Expanding Shared e-Mobility Systems: A Multi-agent Deep Reinforcement Learning Approach
University of Exeter（埃克塞特大学）；The Alan Turing Institute（阿兰·图灵研究所）；University of Warwick（华威大学）；Tongji University（同济大学）；University of Leeds（利兹大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Columbia University（哥伦比亚大学）
Technical University of Denmark（丹麦技术大学）；Technical University of Munich（慕尼黑工业大学）；University of Groningen（格罗宁根大学）；University of Twente（特文特大学）；Munich Data Science Institute（慕尼黑数据科学研究所）
Bike-sharing systems are a rapidly developing mode of transportation and provide an efficient alternative to passive, motorized personal mobility. The asymmetric nature of bike demand causes the need for rebalancing bike stations, which is typically done during night time. To determine the optimal starting inventory level of a station for a given day, a User Dissatisfaction Function (UDF) models user pickups and returns as non-homogeneous Poisson processes with piece-wise linear rates. In this paper, we devise a deep generative model directly applicable in the UDF by introducing a variational …
Microsoft Research（微软研究院）；IBM Research AI（IBM人工智能研究院）；Singapore Management University（新加坡管理大学）
We propose a framework, called neural-progressive hedging (NP), that leverages stochastic programming during the online phase of executing a reinforcement learning (RL) policy. The goal is to ensure feasibility with respect to constraints and risk-based objectives such as conditional value-at-risk (CVaR) during the execution of the policy, using probabilistic models of the state transitions to guide policy adjustments. The framework is particularly amenable to the class of sequential resource allocation problems since feasibility with respect to typical resource constraints cannot be enforced …
GERAD & Department of Decision Sciences, HEC Montréal（蒙特利尔高等商学院决策科学系与GERAD联合机构）；Telfer School of Management, University of Ottawa（渥太华大学特尔弗管理学院）；Evovest（伊沃维斯特）
The problem of portfolio management represents an important and challenging class of dynamic decision making problems, where rebalancing decisions need to be made over time with the consideration of many factors such as investors preferences, trading environments, and market conditions. In this paper, we present a new portfolio policy network architecture for deep reinforcement learning (DRL)that can exploit more effectively cross-asset dependency information and achieve better performance than state-of-the-art architectures. In particular, we introduce a new property, referred to as \textit{a…
Enhancing Deep Hedging of Options with Implied Volatility Surface Feedback Information
Department of Finance, HEC Montréal, Canada（蒙特利尔HEC金融系）；GERAD and Department of Decision Sciences, HEC Montréal, Canada（GERAD与蒙特利尔HEC决策科学系）；Concordia University, Department of Mathematics and Statistics, Canada（康科迪亚大学数学与统计学系）
