Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning

MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning

Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting the visual attention of drivers in accident-prone situations. MEDIRL predicts fixation locations that lead to maximal rewards by learning a task-sensitive reward function from eye fixation patterns recorded from attentive drivers. Additionally, we introduce EyeCar, a new driver attention dataset in accident-prone situations. We conduct comprehensive experiments to evaluate our proposed model on three common benchmark…

Safe Reinforcement Learning for an Energy-Efficient Driver Assistance System

Reinforcement learning (RL)-based driver assistance systems seek to improve fuel consumption via continual improvement of powertrain control actions considering experiential data from the field. However, the need to explore diverse experiences in order to learn optimal policies often limits the application of RL techniques in safety-critical systems like vehicle control. In this paper, an exponential control barrier function (ECBF) is derived and utilized to filter unsafe actions proposed by an RL-based driver assistance system. The RL agent freely explores and optimizes the performance object…

ALIGN-HOLD：滴滴大规模网约车匹配中的实时保持控制的经验对齐

ALIGN-HOLD: Experience Alignment for Real-Time Hold Control in Large-Scale Ride-Hailing Matching at DiDi

ALIGN-HOLD通过从隐式市场偏好中学习保持策略，利用经验对齐框架和奖励模型，在滴滴平台上显著提升行程完成率和司机收入，减少乘客取消。

可扩展的网约车再平衡方法：具有服务可及性保证的约束平均场强化学习

Scalable Ride-Sourcing Vehicle Rebalancing with Service Accessibility Guarantee: A Constrained Mean-Field Reinforcement Learning Approach

提出基于约束平均场强化学习的连续状态再平衡模型，在保证服务公平性的同时实现大规模网约车车队的高效协调与可扩展性。

基于随机效用逆强化学习的轨迹建模

Trajectory Modeling via Random Utility Inverse Reinforcement Learning

本文提出基于随机效用理论的逆强化学习方法，通过扩展状态和确定性最优策略假设对驾驶员轨迹建模，并证明最大熵逆强化学习为其特例，最后用巴西真实轨迹数据验证了贝叶斯推断。

Optimal Passenger-Seeking Policies on E-hailing Platforms Using Markov Decision Process and Imitation Learning

Vacant taxi drivers' passenger seeking process in a road network generates additional vehicle miles traveled, adding congestion and pollution into the road network and the environment. This paper aims to employ a Markov Decision Process (MDP) to model idle e-hailing drivers' optimal sequential decisions in passenger-seeking. Transportation network companies (TNC) or e-hailing (e.g., Didi, Uber) drivers exhibit different behaviors from traditional taxi drivers because e-hailing drivers do not need to actually search for passengers. Instead, they reposition themselves so that the matching platfo…

混合交通中网联与自动驾驶车辆协作变道的多智能体强化学习

Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic

本文针对混合交通中多自动驾驶车辆与人类驾驶车辆共存的变道决策问题，提出了一种基于多智能体优势演员-评论家网络（MA2C）的MARL框架，通过多目标奖励函数优化燃油效率、舒适性和安全性，实验证明其在效率、安全性和舒适性上优于现有基准。

何时可以通过可验证奖励进行组合推理学习？

When Is Compositional Reasoning Learnable from Verifiable Rewards?

本文研究了通过可验证奖励强化学习学习组合推理的可学习性，提出了任务优势比率的概念，并分析了其在不同任务中的表现及影响因素。

超越二元奖励：强化遗忘的奖励设计比较研究

Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning

该研究在RUL框架下提出两种新型奖励函数，经RWKU基准实验，其遗忘效率较二元奖励提升，速度快达3倍且保留模型效用，表明奖励设计是机器遗忘效率的关键驱动因素。

Inverse Reinforcement Learning with Multiple Ranked Experts

We consider the problem of learning to behave optimally in a Markov Decision Process when a reward function is not specified, but instead we have access to a set of demonstrators of varying performance. We assume the demonstrators are classified into one of k ranks, and use ideas from ordinal regression to find a reward function that maximizes the margin between the different ranks. This approach is based on the idea that agents should not only learn how to behave from experts, but also how not to behave from non-experts. We show there are MDPs where important differences in the reward functio…

把握匹配时机：一种面向网约车与合乘服务的深度强化学习方法

Timing the Match: A Deep Reinforcement Learning Approach for Ride-Hailing and Ride-Pooling Services

本文提出基于深度强化学习的自适应网约车匹配时机决策方法，并结合PBRS缓解稀疏奖励，实验证明其可显著降低等待与绕路延误。

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

This study proposes a framework for human-like autonomous car-following planning based on deep reinforcement learning (deep RL). Historical driving data are fed into a simulation environment where an RL agent learns from trial and error interactions based on a reward function that signals how much the agent deviates from the empirical data. Through these interactions, an optimal policy, or car-following model that maps in a human-like way from speed, relative speed between a lead and following vehicle, and inter-vehicle spacing to acceleration of a following vehicle is finally obtained. The mo…

Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving

In the past decades, we have witnessed significant progress in the domain of autonomous driving. Advanced techniques based on optimization and reinforcement learning (RL) become increasingly powerful at solving the forward problem: given designed reward/cost functions, how should we optimize them and obtain driving policies that interact with the environment safely and efficiently. Such progress has raised another equally important question: \emph{what should we optimize}? Instead of manually specifying the reward functions, it is desired that we can extract what human drivers try to optimize …

Making the most of your day: online learning for optimal allocation of time

We study online learning for optimal allocation when the resource to be allocated is time. %Examples of possible applications include job scheduling for a computing server, a driver filling a day with rides, a landlord renting an estate, etc. An agent receives task proposals sequentially according to a Poisson process and can either accept or reject a proposed task. If she accepts the proposal, she is busy for the duration of the task and obtains a reward that depends on the task duration. If she rejects it, she remains on hold until a new task proposal arrives. We study the regret incurred by…

Processing Network Controls via Deep Reinforcement Learning

Novel advanced policy gradient (APG) algorithms, such as proximal policy optimization (PPO), trust region policy optimization, and their variations, have become the dominant reinforcement learning (RL) algorithms because of their ease of implementation and good practical performance. This dissertation is concerned with theoretical justification and practical application of the APG algorithms for solving processing network control optimization problems. Processing network control problems are typically formulated as Markov decision process (MDP) or semi-Markov decision process (SMDP) problems t…

将互信息作为强化学习智能体内在奖励用于按需拼车

Mutual Information as Intrinsic Reward of Reinforcement Learning Agents for On-demand Ride Pooling

针对按需拼车中分布不寻常请求难以获服务的问题，提出基于强化学习的车辆调度框架，将车辆与订单分布间的互信息作为内在奖励以改善相关性，在真实数据集上使收入平均提升3%。

LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning

Multiple-Intent Inverse Reinforcement Learning (MI-IRL) seeks to find a reward function ensemble to rationalize demonstrations of different but unlabelled intents. Within the popular expectation maximization (EM) framework for learning probabilistic MI-IRL models, we present a warm-start strategy based on up-front clustering of the demonstrations in feature space. Our theoretical analysis shows that this warm-start solution produces a near-optimal reward ensemble, provided the behavior modes satisfy mild separation conditions. We also propose a MI-IRL performance metric that generalizes the po…

基于次优专家的逆强化学习

Inverse Reinforcement Learning with Sub-optimal Experts

本文将逆强化学习扩展至包含多个次优专家的场景，研究了可行奖励集的理论性质，证明次优专家能显著缩小兼容奖励集，并提出在次优专家性能接近最优时达到最小最大最优的均匀采样算法。

Formulation and validation of a car-following model based on deep reinforcement learning

We propose and validate a novel car following model based on deep reinforcement learning. Our model is trained to maximize externally given reward functions for the free and car-following regimes rather than reproducing existing follower trajectories. The parameters of these reward functions such as desired speed, time gap, or accelerations resemble that of traditional models such as the Intelligent Driver Model (IDM) and allow for explicitly implementing different driving styles. Moreover, they partially lift the black-box nature of conventional neural network models. The model is trained on …

反事实沙普利信用分配

Counterfactual Shapley Credit Assignment

研究信用分配问题，提出基于因果理论的反事实沙普利信用分配框架，通过反事实沙普利值分配功劳责任，推导有效估计器实现$\phi$-PPO方法，结合优先轨迹重放，在复杂环境中有卓越样本效率，能精确对齐奖励真实原因。

深度强化学习中的内在代偿条件

Intrinsic Vicarious Conditioning for Deep Reinforcement Learning

本文提出一种内在代偿条件机制，通过记忆方法实现注意力、保留、再现和强化四个步骤，支持低样本学习，无需示范者策略或奖励函数，在MiniWorld Sidewalk和Box2D CarRacing环境中验证了其效果。

RDAR：面向自动驾驶的奖励驱动智能体相关性估计

RDAR: Reward-Driven Agent Relevance Estimation for Autonomous Driving

针对自动驾驶场景中智能体过多导致计算开销大的问题，本文提出RDAR方法，通过强化学习学习每个智能体的相关性并生成掩码以排除无关智能体，在保持驾驶性能的同时显著减少处理数量。

Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving

A model used for velocity control during car following was proposed based on deep reinforcement learning (RL). To fulfil the multi-objectives of car following, a reward function reflecting driving safety, efficiency, and comfort was constructed. With the reward function, the RL agent learns to control vehicle speed in a fashion that maximizes cumulative rewards, through trials and errors in the simulation environment. A total of 1,341 car-following events extracted from the Next Generation Simulation (NGSIM) dataset were used to train the model. Car-following behavior produced by the model wer…

Equilibrium Inverse Reinforcement Learning for Ride-hailing Vehicle Network

Ubiquitous mobile computing have enabled ride-hailing services to collect vast amounts of behavioral data of riders and drivers and optimize supply and demand matching in real time. While these mobility service providers have some degree of control over the market by assigning vehicles to requests, they need to deal with the uncertainty arising from self-interested driver behavior since workers are usually free to drive when they are not assigned tasks. In this work, we formulate the problem of passenger-vehicle matching in a sparsely connected graph and proposed an algorithm to derive an equi…

iPLAN：通过分布式多智能体强化学习在异构交通中实现意图感知规划

iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning

本文提出分布式 MARL 方法 iPLAN，使自动驾驶智能体仅凭局部观测推断其他驾驶员的行为与即时意图，并在两类交通仿真中显著提升奖励、成功率和生存时间。

GARLIC：基于GPT增强强化学习与智能控制的车辆调度方法

GARLIC: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching

针对网约车调度中交通动态复杂、供需失衡等问题，提出GARLIC框架，结合多视图图、动态奖励函数与GPT模型，实现高精度预测与策略优化，实验证明可降低空载率并适配驾驶员行为。

Optimizing Taxi Carpool Policies via Reinforcement Learning and Spatio-Temporal Mining

In this paper, we develop a reinforcement learning (RL) based system to learn an effective policy for carpooling that maximizes transportation efficiency so that fewer cars are required to fulfill the given amount of trip demand. For this purpose, first, we develop a deep neural network model, called ST-NN (Spatio-Temporal Neural Network), to predict taxi trip time from the raw GPS trip data. Secondly, we develop a carpooling simulation environment for RL training, with the output of ST-NN and using the NYC taxi trip dataset. In order to maximize transportation efficiency and minimize traffic …

LLMs中RL越狱的系统性研究

A Systematic Investigation of RL-Jailbreaking in LLMs

本文首次系统分解RL越狱框架，通过分析奖励函数、动作空间、回合长度等环境形式化因素和算法措施，发现密集奖励和延长回合长度是越狱成功的主要驱动因素，并提供了提升RL越狱效率及强化模型防御的工具。

Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms

We provide new perspectives and inference algorithms for Maximum Entropy (MaxEnt) Inverse Reinforcement Learning (IRL), which provides a principled method to find a most non-committal reward function consistent with given expert demonstrations, among many consistent reward functions. We first present a generalized MaxEnt formulation based on minimizing a KL-divergence instead of maximizing an entropy. This improves the previous heuristic derivation of the MaxEnt IRL model (for stochastic MDPs), allows a unified view of MaxEnt IRL and Relative Entropy IRL, and leads to a model-free learning alg…

Learning to Drive in a Day

We demonstrate the first application of deep reinforcement learning to autonomous driving. From randomly initialised parameters, our model is able to learn a policy for lane following in a handful of training episodes using a single monocular image as input. We provide a general and easy to obtain reward: the distance travelled by the vehicle without the safety driver taking control. We use a continuous, model-free deep reinforcement learning algorithm, with all exploration and optimisation performed on-vehicle. This demonstrates a new framework for autonomous driving which moves away from rel…

强化学习中的统计推断：选择性综述

Statistical Inference in Reinforcement Learning: A Selective Survey

本文选择性综述强化学习（RL）中的统计推断工具，涵盖假设检验与置信区间构建，强调其对统计学和机器学习社区的价值，推动经典统计推断工具在RL领域的应用。

用强化学习建模客户轨迹以获得实际零售洞察

Modelling Customer Trajectories with Reinforcement Learning for Practical Retail Insights

本文提出了一种基于智能体的建模框架，将客户轨迹预测转化为最大熵强化学习问题，以更准确地反映具有有限理性的客户行为，从而提供更精确的冲动购买率和货架交通密度估计。

A Distributed Model-Free Ride-Sharing Approach for Joint Matching, Pricing, and Dispatching using Deep Reinforcement Learning

Significant development of ride-sharing services presents a plethora of opportunities to transform urban mobility by providing personalized and convenient transportation while ensuring efficiency of large-scale ride pooling. However, a core problem for such services is route planning for each driver to fulfill the dynamically arriving requests while satisfying given constraints. Current models are mostly limited to static routes with only two rides per vehicle (optimally) or three (with heuristics). In this paper, we present a dynamic, demand aware, and pricing-based vehicle-passenger matching…

Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarios

When learning to behave in a stochastic environment where safety is critical, such as driving a vehicle in traffic, it is natural for human drivers to plan fallback strategies as a backup to use if ever there is an unexpected change in the environment. Knowing to expect the unexpected, and planning for such outcomes, increases our capability for being robust to unseen scenarios and may help prevent catastrophic failures. Control of Autonomous Vehicles (AVs) has a particular interest in knowing when and how to use fallback strategies in the interest of safety. Due to imperfect information avail…

A Closer Look at the Worst-case Behavior of Multi-armed Bandit Algorithms

One of the key drivers of complexity in the classical (stochastic) multi-armed bandit (MAB) problem is the difference between mean rewards in the top two arms, also known as the instance gap. The celebrated Upper Confidence Bound (UCB) policy is among the simplest optimism-based MAB algorithms that naturally adapts to this gap: for a horizon of play n, it achieves optimal O(log n) regret in instances with "large" gaps, and a near-optimal O(\sqrt{n log n}) minimax regret when the gap can be arbitrarily "small." This paper provides new results on the arm-sampling behavior of UCB, leading to seve…

Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning

Autonomous car racing is a major challenge in robotics. It raises fundamental problems for classical approaches such as planning minimum-time trajectories under uncertain dynamics and controlling the car at the limits of its handling. Besides, the requirement of minimizing the lap time, which is a sparse objective, and the difficulty of collecting training data from human experts have also hindered researchers from directly applying learning-based approaches to solve the problem. In the present work, we propose a learning-based system for autonomous car racing by leveraging a high-fidelity phy…

LongAct：利用内在激活模式促进长上下文强化学习

LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning

本文提出LongAct方法，通过利用长上下文推理中的稀疏结构，改进大语言模型的训练，提升长上下文处理能力与泛化性能。

通过强化学习实现全向轮椅的共享控制

Shared Control of Holonomic Wheelchairs through Reinforcement Learning

针对全向轮椅共享控制中行为不直观、未发挥全向能力的问题，提出强化学习方法将二维用户输入映射为三维运动，并通过仿真与实机验证其安全、平滑且低认知负荷。

通过逆强化学习理解电力消费行为

Understanding electricity consumption behaviour through Inverse Reinforcement Learning

研究将家庭视为与复杂环境交互的主体，用逆强化学习将其消费行为表示为模型隐含奖励函数，观察奖励函数在社会经济和气候冲击下的变化，测试框架并发现冲击对不同消费群体冷却行为的异质影响。

Value Variance Minimization for Learning Approximate Equilibrium in Aggregation Systems

For effective matching of resources (e.g., taxis, food, bikes, shopping items) to customer demand, aggregation systems have been extremely successful. In aggregation systems, a central entity (e.g., Uber, Food Panda, Ofo) aggregates supply (e.g., drivers, delivery personnel) and matches demand to supply on a continuous basis (sequential decisions). Due to the objective of the central entity to maximize its profits, individual suppliers get sacrificed thereby creating incentive for individuals to leave the system. In this paper, we consider the problem of learning approximate equilibrium soluti…

Application of Deep Reinforcement Learning to Payment Fraud

The large variety of digital payment choices available to consumers today has been a key driver of e-commerce transactions in the past decade. Unfortunately, this has also given rise to cybercriminals and fraudsters who are constantly looking for vulnerabilities in these systems by deploying increasingly sophisticated fraud attacks. A typical fraud detection system employs standard supervised learning methods where the focus is on maximizing the fraud recall rate. However, we argue that such a formulation can lead to sub-optimal solutions. The design requirements for these fraud models require…

ReVeal：通过可靠自验证实现自我进化的代码智能体

ReVeal: Self-Evolving Code Agents via Reliable Self-Verification

本文提出 ReVeal 多轮强化学习框架，通过显式优化自验证、工具评估和 TAPO 轮次信用分配，推动代码与测试协同进化，显著提升 LiveCodeBench 上的长程改进能力和 Pass@k。

RAST-MoE-RL: 一种面向动态的时空MoE框架用于网约车深度强化学习

RAST-MoE-RL: A Regime-Aware Spatio-Temporal MoE Framework for Deep Reinforcement Learning in Ride-Hailing

RAST-MoE-RL通过引入制度感知的时空MoE框架，提升了网约车深度强化学习中的决策效率与稳定性，实现了更高的奖励和更短的等待时间。

利他性拼车：通过点对点激励实现公平可持续的城市交通框架

Altruistic Ride Sharing: A Framework for Fair and Sustainable Urban Mobility via Peer-to-Peer Incentives

本文提出利他性拼车框架，通过去中心化的点对点激励机制，减少交通拥堵和碳排放，提高车辆利用率。

用于自动驾驶大学生方程式赛车（F:SAE）车辆局部路径跟踪的深度强化学习

Deep Reinforcement Learning for Local Path Following of an Autonomous Formula SAE Vehicle

本文提出利用深度强化学习和逆向强化学习将局部锥桶位置映射为转向角以实现赛道跟踪，采用SAC和AIRL算法及三种新型奖励函数在仿真与现实中成功实现了局部路径跟踪，并提出了向完整F:SAE车辆扩展的未来建议。

PACEvolve++: 提高进化搜索代理的测试时学习

PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents

PACEvolve++通过强化学习框架提升进化搜索代理的测试时策略适应能力，采用可训练顾问生成并评估假设，结合前沿模型生成可执行候选，实现更快收敛和更稳定的测试时训练。

基于深度强化学习的自动驾驶决策策略研究

Research on Autonomous Driving Decision-making Strategies based Deep Reinforcement Learning

本文研究自动驾驶行为决策问题，采用DQN和PPO两种深度强化学习方法并改进奖励函数设计，实验证明其决策策略在多种驾驶任务中优于传统基于规则的方法。

GRPO在离线策略下的可能性：Mu-GRPO用于高效的大语言模型强化学习

How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning

本文探讨了GRPO在离线策略下的可行性，提出Mu-GRPO方法，通过减少rollout-optimization切换开销，实现高效的LLM强化学习，同时在多个基准测试中表现出色。

基于图神经网络强化学习的交通感知出租车最优布置

Traffic-Aware Optimal Taxi Placement Using Graph Neural Network-Based Reinforcement Learning

本文提出基于图神经网络强化学习的交通感知出租车最优布置方法，通过实时整合交通数据优化出租车热点，有效减少乘客等待时间和司机行驶距离。

从P(y|x)到P(y)：在预训练空间中研究强化学习

From $P(y|x)$ to $P(y)$: Investigating Reinforcement Learning in Pre-train Space

本文提出PreRL和DSRL方法，通过优化预训练空间中的边际分布P(y)，提升LLM推理能力，并通过NSR机制增强推理效果，实验表明DSRL在推理任务中表现优异。

Generative Personas That Behave and Experience Like Humans

Using artificial intelligence (AI) to automatically test a game remains a critical challenge for the development of richer and more complex game worlds and for the advancement of AI at large. One of the most promising methods for achieving that long-standing goal is the use of generative AI agents, namely procedural personas, that attempt to imitate particular playing behaviors which are represented as rules, rewards, or human demonstrations. All research efforts for building those generative agents, however, have focused solely on playing behavior which is arguably a narrow perspective of wha…

并非所有标记都必要（NAT）：标记效率强化学习

Not all tokens are needed(NAT): token efficient reinforcement learning

NAT通过局部标记采样提升RL效率，使用50%标记实现与完整标记相同性能。

重新审视强化学习在大推理模型中的熵

Revisiting Entropy in Reinforcement Learning for Large Reasoning Models

本文研究了强化学习可验证奖励中大语言模型熵崩溃问题，分析了熵与响应多样性、校准及性能的关系，提出正优势重加权方法以调节熵并保持性能。

多目标强化学习用于高速公路卡车战术决策

Preference-Conditioned Multi-Objective Reinforcement Learning for Tactical Decision Making in Heavy-Duty Vehicles

提出基于近端策略优化的多目标强化学习框架，学习一组帕累托最优策略以平衡安全性、能源效率和时间效率，实现无需重新训练的灵活决策。