消退的专家引导：桥接基于模型与基于学习的控制在可中止自主超车中的应用
Fading Expert Guidance: Bridging Model-Based and Learning-Based Control for Abortable Autonomous Overtaking
AI总结：
提出消退专家引导框架，用基于模型的控制器引导DRL训练，提升可中止超车任务的样本效率和安全性，且不限制最终策略为模仿。
摘要：
在双车道公路上超车是自动驾驶汽车面临的一个安全关键决策问题，因为对向来车可能迫使自车中止超车并并回原车道。深度强化学习（DRL）在此类连续控制任务中具有前景，但需要大量的交互数据，并且训练期间可能产生不安全的探索动作。基于模型的控制器提供结构化的行为，但依赖于模型的准确性和手工设计的逻辑。本文提出了一种消退的专家引导框架，利用基于模型的控制器来引导自主超车的DRL训练。该专家结合了约束迭代LQR（CiLQR）规划器与基于PID的辅助控制器，当优化计划变得不可行时激活辅助控制器。专家动作通过一个消退项进入演员目标，该消退项被解释为围绕专家策略的时变软信任区域。该术语首先将策略偏向专家，然后消失，允许根据强化学习目标进行优化。该方法使用PPO、TD3和SAC进行评估。双向交通仿真显示样本效率和最终任务性能均有提升。安全效果依赖于算法：引导减少了SAC的车辆碰撞，消除了TD3的边界碰撞，而PPO的碰撞情况保持不变。总体而言，结果支持消退专家引导作为一种实用机制，用于将基于模型的驾驶知识转移到基于学习的控制器，而不会在具有双向交通的可中止超车任务中将最终策略限制为模仿。
英文摘要：
Overtaking on two-lane roads is a safety-critical decision-making problem for autonomous vehicles, since oncoming traffic may force the ego vehicle to abort and merge back to its original lane. Deep reinforcement learning (DRL) is promising for such continuous-control tasks, but it requires substantial interaction data and may produce unsafe exploratory actions during training. Model-based controllers provide structured behavior, but depend on model fidelity and hand-designed logic. This paper proposes a fading expert-guidance framework that uses a model-based controller to guide DRL training for autonomous overtaking. The expert combines a constrained iterative LQR (CiLQR) planner with PID-based auxiliary controllers, activated when the optimized plan becomes infeasible. The expert action enters the actor objective through a fading term, interpreted as a time-varying soft trust region around the expert policy. This term first biases the policy toward the expert and then vanishes, allowing optimization according to the reinforcement-learning objective. The method is evaluated with PPO, TD3, and SAC. Simulations with bidirectional traffic show improved sample efficiency and final task performance. The safety effects are algorithm-dependent: guidance reduces vehicle collisions for SAC, eliminates boundary collisions for TD3, and leaves the PPO collision profile unchanged. Overall, the results support fading expert guidance as a practical mechanism for transferring model-based driving knowledge to learning-based controllers without restricting the final policy to imitation in an abortable overtaking task with bidirectional traffic.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
