Online Decision-Making Under Uncertainty for Vehicle-to-Building Systems
Vanderbilt University（范德比大学）；Nissan Advanced Technology Center - Silicon Valley（日产先进技术中心-硅谷）
Rishav Sen, Yunuo Zhang, Fangqi Liu, Jose Paolo Talusan, Ava Pettet, Yoshinori Suzue, Ayan Mukhopadhyay, Abhishek Dubey
备注：
17 pages, 2 figures, 10 tables. Published in the Proceedings of the 16th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS '25), May 06--09, 2025, Irvine, CA, USA
发表信息：
Proceedings of 16th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS), 2025
Vehicle-to-building (V2B) systems integrate physical infrastructures, such as smart buildings and electric vehicles (EVs) connected to chargers at the building, with digital control mechanisms to manage energy use. By utilizing EVs as flexible energy reservoirs, buildings can dynamically charge and discharge them to optimize energy use and cut costs under time-variable pricing and demand charge policies. This setup leads to the V2B optimization problem, where buildings coordinate EV charging and discharging to minimize total electricity costs while meeting users' charging requirements. However, the V2B optimization problem is challenging because of: (1) fluctuating electricity pricing, which includes both energy charges ($/kWh) and demand charges ($/kW); (2) long planning horizons (typically over 30 days); (3) heterogeneous chargers with varying charging rates, controllability, and directionality (i.e., unidirectional or bidirectional); and (4) user-specific battery levels at departure to ensure user requirements are met. In contrast to existing approaches that often model this setting as a single-shot combinatorial optimization problem, we highlight critical limitations in prior work and instead model the V2B optimization problem as a Markov decision process (MDP), i.e., a stochastic control process. Solving the resulting MDP is challenging due to the large state and action spaces. To address the challenges of the large state space, we leverage online search, and we counter the action space by using domain-specific heuristics to prune unpromising actions. We validate our approach in collaboration with Nissan Advanced Technology Center - Silicon Valley. Using data from their EV testbed, we show that the proposed framework significantly outperforms state-of-the-art methods.
The Ohio State University（俄亥俄州立大学）；Ford Greenfield Labs（福特格林菲尔德实验室）；Ford Motor Company（福特汽车公司）
In this paper, an online evolving framework is proposed to detect and revise a controller's imperfect decision-making in advance. The framework consists of three modules: the evolving Finite State Machine (e-FSM), action-reviser, and controller modules. The e-FSM module evolves a stochastic model (e.g., Discrete-Time Markov Chain) from scratch by determining new states and identifying transition probabilities repeatedly. With the latest stochastic model and given criteria, the action-reviser module checks validity of the controller's chosen action by predicting future states. Then, if the chos…
National University of Defense Technology（国防科技大学）；Tsinghua University（清华大学）
The unmanned aerial vehicle (UAV) plays an vital role in various applications such as delivery, military mission, disaster rescue, communication, etc., due to its flexibility and versatility. This paper proposes a deep reinforcement learning method to solve the UAV online routing problem with wireless power transfer, which can charge the UAV remotely without wires, thus extending the capability of the battery-limited UAV. Our study considers the power consumption of the UAV and the wireless charging process. Unlike the previous works, we solve the problem by a designed deep neural network. The…
This paper describes important considerations and challenges associated with online reinforcement-learning based waveform selection for target identification in frequency modulated continuous wave (FMCW) automotive radar systems. We present a novel learning approach based on satisficing Thompson sampling, which quickly identifies a waveform expected to yield satisfactory classification performance. We demonstrate through measurement-level simulations that effective waveform selection strategies can be quickly learned, even in cases where the radar must select from a large catalog of candidate …
Huawei Technologies（华为技术有限公司）；Université de Sherbrooke（舍布鲁克大学）
Motion planning under uncertainty is one of the main challenges in developing autonomous driving vehicles. In this work, we focus on the uncertainty in sensing and perception, resulted from a limited field of view, occlusions, and sensing range. This problem is often tackled by considering hypothetical hidden objects in occluded areas or beyond the sensing range to guarantee passive safety. However, this may result in conservative planning and expensive computation, particularly when numerous hypothetical objects need to be considered. We propose a reinforcement learning (RL) based solution to…
A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE（知识驱动的人机智能工程研究中心，教育部）；International Center of Future Science, Jilin University（未来科学国际中心，吉林大学）；JPMorgan AI Research（摩根大通AI研究）；Northwestern Polytechnical University（西北工业大学）；Lehigh University（莱斯大学）
We extend the adversarial/non-stochastic multi-play multi-armed bandit (MPMAB) to the case where the number of arms to play is variable. The work is motivated by the fact that the resources allocated to scan different critical locations in an interconnected transportation system change dynamically over time and depending on the environment. By modeling the malicious hacker and the intrusion monitoring system as the attacker and the defender, respectively, we formulate the problem for the two players as a sequential pursuit-evasion game. We derive the condition under which a Nash equilibrium of…
Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Tsinghua University（清华大学）
Cooperative perception of connected vehicles comes to the rescue when the field of view restricts stand-alone intelligence. While raw-level cooperative perception preserves most information to guarantee accuracy, it is demanding in communication bandwidth and computation power. Therefore, it is important to schedule the most beneficial vehicle to share its sensor in terms of supplementary view and stable network connection. In this paper, we present a model of raw-level cooperative perception and formulate the energy minimization problem of sensor sharing scheduling as a variant of the Multi-A…
Efficient and Uncertainty-Aware Diffusion Framework for Offline-to-Online Reinforcement Learning
Department of Computer Science, Johns Hopkins University, Baltimore, MD, U.S.A.（约翰霍普金斯大学计算机科学系）；AMLab, University of Amsterdam, Amsterdam, Netherlands（阿姆斯特丹大学AM实验室）
Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness
Northwestern Polytechnical University（西北工业大学）；Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong University of Science and Technology（香港科技大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shenzhen Research Institute of Northwestern Polytechnical University（西北工业大学深圳研究院）
In this paper, we study an online charge scheduling strategy for fleets of autonomous-mobility-on-demand electric vechicles (AMoD EVs). We consider the case where vehicles complete trips and then enter a between-ride state throughout the day, with their information becoming available to the fleet operator in an online fashion. In the between-ride state, the vehicles must be scheduled for charging and then routed to their next passenger pick-up locations. Additionally, due to the unknown daily sequences of ride requests, the problem cannot be solved by any offline approach. As such, we study an…
University of Pennsylvania（宾夕法尼亚大学）；Instituto de Ciencias Matematicas (CSIC-UAM-UCM-UC3M)（数学科学研究所（西班牙国家研究委员会-马德里自治大学-马德里康普顿斯大学-卡洛斯三世大学））；Technical University of Munich（慕尼黑工业大学）
Underactuated vehicles have gained much attention in the recent years due to the increasing amount of aerial and underwater vehicles as well as nanosatellites. Trajectory tracking control of these vehicles is a substantial aspect for an increasing range of application domains. However, external disturbances and parts of the internal dynamics are often unknown or very time-consuming to model. To overcome this issue, we present a tracking control law for underactuated rigid-body dynamics using an online learning-based oracle for the prediction of the unknown dynamics. We show that Gaussian proce…
The cooperative control applied to vehicles allows the optimization of traffic on the roads. There are many aspects to consider in the case of the operation of autonomous vehicles on highways since there are different external parameters that can be involved in the analysis of a network. In this paper, we present the design and simulation of adaptive control for a platoon with heterogeneous vehicles, taking into account that not all vehicles can communicate their control input, and in turn include structured nonlinear uncertainty input parameters.
In order to increase the prediction accuracy of the online vehicle velocity prediction (VVP) strategy, a self-adaptive velocity prediction algorithm fused with traffic information was presented for the multiple scenarios. Initially, traffic scenarios were established inside the co-simulation environment. In addition, the algorithm of a general regressive neural network (GRNN) paired with datasets of the ego-vehicle, the front vehicle, and traffic lights was used in traffic scenarios, which increasingly improved the prediction accuracy. To ameliorate the robustness of the algorithm, then the st…
Automated vehicles need to be aware of the capabilities they currently possess. Skill graphs are directed acylic graphs in which a vehicle's capabilities and the dependencies between these capabilities are modeled. The skills a vehicle requires depend on the behaviors the vehicle has to perform and the operational design domain (ODD) of the vehicle. Skill graphs were originally proposed for online monitoring of the current capabilities of an automated vehicle. They have also been shown to be useful during other parts of the development process, e.g. system design, system verification. Skill gr…
This paper presents a novel algorithm, called $ε^*$+, for online coverage path planning of unknown environments using energy-constrained autonomous vehicles. Due to limited battery size, the energy-constrained vehicles have limited duration of operation time. Therefore, while executing a coverage trajectory, the vehicle has to return to the charging station for a recharge before the battery runs out. In this regard, the $ε^*$+ algorithm enables the vehicle to retreat back to the charging station based on the remaining energy which is monitored throughout the coverage process. This is followed …
Regulatory approval and safety guarantees for autonomous vehicles facing frequent functional updates and complex software stacks, including artificial intelligence, are a challenging topic. This paper proposes a concept and guideline for the development of an online verification module -- the Supervisor -- capable of handling the aforementioned challenges. The concept presented for the establishment of a Supervisor is designed in a way to identify and monitor an extensive list of features contributing to safe operation. As a result, a safe overall (sub)system is attained. Safeguarding a motion…
Physics-informed deep learning is a popular trend in the modeling and control of dynamical systems. This paper presents a novel method for rapid online identification of vehicle cornering stiffness coefficient, a crucial parameter in vehicle stability control models and control algorithms. The new method enables designers to rapidly identify the vehicle front and rear cornering stiffness parameters so that the controller reference gains can be re-adjusted under varying road and vehicle conditions to improve the reference tracking performance of the control system during operation. The proposed…
Ensuring the safety of autonomous vehicles, given the uncertainty in sensing other road users, is an open problem. Moreover, separate safety specifications for perception and planning components raise how to assess the overall system safety. This work provides a probabilistic approach to calculate safety envelopes under perception uncertainty. The probabilistic envelope definition is based on a risk threshold. It limits the cumulative probability that the actual safety envelope in a fully observable environment is larger than an applied envelope and is solved using iterative worst-case analysi…
We study the feasibility of using electric vehicles in online, high-capacity ridepooling systems. Prior work has shown that online algorithms perform well for centrally-controlled, high-capacity ridepool systems. First, we propose a mixed integer linear program to expand past algorithms on ridepooling to electric vehicles fleets with the objective of scheduling vehicle charging to maintain sufficient fleet sizes at various times of day. Then we show a faster, scalable algorithm with similar performance that is practical for full-scale systems. Our contributions show the importance of having kn…
Shift of the power system generation from the fossil to the variable renewable sources prompted the system operators to search for new sources of flexibility, that is, new reserve providers. With the introduction of electric vehicles, smart charging emerged as one of the relevant solutions. However, electric vehicle aggregators face the uncertainty of reserve activation on one side and electric vehicle availability on the other. These uncertainty can have a negative effect on both the aggregators' profitability and their users' comfort.State-of-the art literature mostly neglects the reserve ac…
Nonlinear Robust Model Predictive Control (RMPC) provides a very promising solution to the problem of automatic emergency maneuvering, which is capable of handling multiple possibly conflicting objectives of robustness and performance. Even though RMPC gives a suboptimal solution, the key challenge in real-time implementation is that it is computationally very demanding. In this paper a real-time capable robust tube MPC based framework for steering control during emergency obstacle avoidance maneuver is presented. The novelty of this framework lies in the robust integration of path planning an…
This paper proposes a methodology to estimate uncertainty in automated vehicle (AV) dynamics in real time via Bayesian inference. Based on the estimated uncertainty, the method aims to continuously monitor the car-following (CF) performance of the AV to support strategic actions to maintain a desired performance. Our methodology consists of three sequential components: (i) the Stochastic Gradient Langevin Dynamics (SGLD) is adopted to estimate parameter uncertainty relative to vehicular dynamics in real time, (ii) dynamic monitoring of car-following stability (local and string-wise), and (iii)…
Online Location Planning for AI-Defined Vehicles: Optimizing Joint Tasks of Order Serving and Spatio-Temporal Heterogeneous Model Fine-Tuning
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Peng Cheng Laboratory（鹏城实验室）
We pose the aggregator's problem as a bilevel model, where the upper level minimizes the total operation costs of the fleet of EVs, while each lower level minimizes the energy available to each vehicle for transportation given a certain charging plan. Thanks to the totally unimodular character of the constraint matrix in the lower-level problems, the model can be mathematically recast as a computationally efficient mixed-integer program that delivers charging schedules that are robust against the uncertain availability of the EVs. Finally, we use synthetic data from the National Household Trav…
In this research, a new data mining-based design approach has been developed for designing complex mechanical systems such as a crashworthy passenger car with uncertainty modeling. The method allows exploring the big crash simulation dataset to design the vehicle at multi-levels in a top-down manner (main energy absorbing system, components, and geometric features) and derive design rules based on the whole vehicle body safety requirements to make decisions towards the component and sub-component level design. Full vehicle and component simulation datasets are mined to build decision trees, wh…
We study the problem of allocating Electric Vehicles (EVs) to charging stations and scheduling their charging. We develop offline and online solutions that treat EV users as self-interested agents that aim to maximise their profit and minimise the impact on their schedule. We formulate the problem of the optimal EV to charging station allocation as a Mixed Integer Programming (MIP) one and we propose two pricing mechanisms: A fixed-price one, and another that is based on the well known Vickrey-Clark-Groves (VCG) mechanism. Later, we develop online solutions that incrementally call the MIP-base…
Ulsan National Institute of Science and Technology（蔚山国立科学技术院）
This paper presents a safe, efficient, and agile ground vehicle navigation algorithm for 3D off-road terrain environments. Off-road navigation is subject to uncertain vehicle-terrain interactions caused by different terrain conditions on top of 3D terrain topology. The existing works are limited to adopt overly simplified vehicle-terrain models. The proposed algorithm learns the terrain-induced uncertainties from driving data and encodes the learned uncertainty distribution into the traversability cost for path evaluation. The navigation path is then designed to optimize the uncertainty-aware …
In this work, a terrain estimation framework is developed for autonomous vehicles operating on deformable terrains. Previous work in this area usually relies on steady state tire operation, linearized classical terramechanics models, or on computationally expensive algorithms that are not suitable for real-time estimation. To address these shortcomings, this work develops a reduced-order nonlinear terramechanics model as a surrogate of the Soil Contact Model (SCM) through extending a state-of-the-art Bekker model to account for additional dynamic effects. It is shown that this reduced-order su…
This paper introduces a robust optimal green light speed advisory system for fixed and actuated traffic signals when a probability distribution is provided. These distributions represent the domain of possible switching times from the Signal Phasing and Timing (SPaT) messages. The system finds the least-cost vehicle trajectory using a computationally efficient A-star algorithm incorporated in a dynamic programming procedure to minimize the vehicle's total fuel consumption. Constraints are introduced to ensure that vehicles do not, collide with other vehicles, run red lights, or exceed a maximu…
