We study learning dynamics induced by myopic travelers who repeatedly play a routing game on a transportation network with an unknown state. The state impacts cost functions of one or more edges of the network. In each stage, travelers choose their routes according to Wardrop equilibrium based on public belief of the state. This belief is broadcast by an information system that observes the edge loads and realized costs on the used edges, and performs a Bayesian update to the prior stage's belief. We show that the sequence of public beliefs and edge load vectors generated by the repeated play converge almost surely. In any rest point, travelers have no incentive to deviate from the chosen routes and accurately learn the true costs on the used edges. However, the costs on edges that are not used may not be accurately learned. Thus, learning can be incomplete in that the edge load vectors at rest point and complete information equilibrium can be different. We present some conditions for complete learning and illustrate situations when such an outcome is not guaranteed.
Ride-sharing services are gaining popularity and are crucial for a sustainable environment. A special case in which such services are most applicable, is the last mile variant. In this variant it is assumed that all the passengers are positioned at the same origin location (e.g. an airport), and each have a different destination. One of the major issues in a shared ride is fairly splitting of the ride cost among the passengers. In this paper we use the Shapley value, which is one of the most significant solution concepts in cooperative game theory, for fairly splitting the cost of a shared rid…
This paper aims to develop a paradigm that models the learning behavior of intelligent agents (including but not limited to autonomous vehicles, connected and automated vehicles, or human-driven vehicles with intelligent navigation systems where human drivers follow the navigation instructions completely) with a utility-optimizing goal and the system's equilibrating processes in a routing game among atomic selfish agents. Such a paradigm can assist policymakers in devising optimal operational and planning countermeasures under both normal and abnormal circumstances. To this end, we develop a M…
Traffic congestion games abstract away from the costs of junctions in transport networks, yet, in urban environments, these often impact journey times significantly. In this paper we equip congestion games with traffic lights, modelled as junction-based waiting cycles, therefore enabling more realistic route planning strategies. Using the SUMO simulator, we show that our modelling choices coincide with realistic routing behaviours, in particular, that drivers' decisions about route choices are based on the proportion of red light time for their direction of travel. Drawing upon the experimenta…
The recent emergence of navigational tools has changed traffic patterns and has now enabled new types of congestion-aware routing control like dynamic road pricing. Using the fundamental diagram of traffic flows - applied in macroscopic and mesoscopic traffic modeling - the article introduces a new N-player dynamic routing game with explicit congestion dynamics. The model is well-posed and can reproduce heterogeneous departure times and congestion spill back phenomena. However, as Nash equilibrium computations are PPAD-complete, solving the game becomes intractable for large but realistic numb…
Game-Theoretic and Algorithmic Analyses of Multi-Agent Routing under Crossing Costs
Kyushu University（九州大学）；Computer Science Institute, Faculty of Mathematics and Physics, Charles University（查理大学数学物理学院计算机科学研究所）；Nagoya University（名古屋大学）
Route controlled autonomous vehicles could have a significant impact in reducing congestion in the future. Before applying multi-agent reinforcement learning algorithms to route control, we can model the system using a congestion game to predict and mitigate potential issues. We consider the problem of distributed operating systems in a transportation network that control the routing choices of their assigned vehicles. We formulate an associated network control game, consisting of multiple actors seeking to optimise the social welfare of their assigned subpopulations in an underlying nonatomic…
Fraunhofer Center for Machine Learning（弗劳恩霍夫机器学习中心）；Fraunhofer IAIS（弗劳恩霍夫智能分析与信息系统研究所）；University of Bonn（波恩大学）
We interpret solving the multi-vehicle routing problem as a team Markov game with partially observable costs. For a given set of customers to serve, the playing agents (vehicles) have the common goal to determine the team-optimal agent routes with minimal total cost. Each agent thereby observes only its own cost. Our multi-agent reinforcement learning approach, the so-called multi-agent Neural Rewriter, builds on the single-agent Neural Rewriter to solve the problem by iteratively rewriting solutions. Parallel agent action execution and partial observability require new rewriting rules for the…
We consider the problem of learning to play a repeated multi-agent game with an unknown reward function. Single player online learning algorithms attain strong regret bounds when provided with full information feedback, which unfortunately is unavailable in many real-world scenarios. Bandit feedback alone, i.e., observing outcomes only for the selected action, yields substantially worse performance. In this paper, we consider a natural model where, besides a noisy measurement of the obtained reward, the player can also observe the opponents' actions. This feedback model, together with a regula…
In this paper we tackle the problem of routing multiple agents in a coordinated manner. This is a complex problem that has a wide range of applications in fleet management to achieve a common goal, such as mapping from a swarm of robots and ride sharing. Traditional methods are typically not designed for realistic environments hich contain sparsely connected graphs and unknown traffic, and are often too slow in runtime to be practical. In contrast, we propose a graph neural network based model that is able to perform multi-agent routing based on learned value iteration in a sparsely connected …
We evaluate the robustness of agents' traffic equilibria in randomized routing games characterized by an uncertain network demand with a possibly unknown probability distribution. Specifically, we extend the so-called hose model by considering a traffic equilibrium model where the uncertain network demand configuration belongs to a polyhedral set, whose shape is itself a-priori unknown. By exploiting available data, we apply the scenario approach theory to establish distribution-free feasibility guarantees for agents' traffic equilibria of the uncertain routing game without the need to know an…
