基于图像的深度强化学习动作平滑正则化在自主微型赛车中的应用
Image-based Regularization for Action Smoothness in Autonomous Miniature Racing Car with Deep Reinforcement Learning
AI总结：
本文提出基于图像的动作平滑正则化（I-RAS）和IR控制，解决自主微型赛车深度强化学习中的抖动控制问题，将成功率从59%提升至95%，并在真实赛道和2022年AWS DeepRacer总决赛中验证了有效性。
摘要：
深度强化学习在低层控制任务中已取得显著成果。然而，对于自动驾驶和无人机飞行等应用，智能体可能突然改变动作，导致控制行为难以稳定，这通常会降低控制系统的效率、引起过度机械磨损，并对车辆造成不可控的危险行为。最近，一种名为动作策略平滑条件化（CAPS）的方法被提出，用于解决四旋翼无人机等应用中低维特征的抖动问题。为应对高维特征，本文提出了基于图像的动作平滑正则化（I-RAS），用于解决自主微型赛车中的抖动控制问题。我们还引入了一种基于影响比的控制方法，即一种自适应正则化权重来控制平滑约束，称为IR控制。在实验中，采用I-RAS和IR控制的智能体将成功率从59%显著提高到95%。在真实世界赛道实验中，该智能体也优于其他方法，即在没有真实世界训练的情况下，平均完成圈速时间更短，同时完成率也更高。基于I-RAS的智能体赢得2022年AWS DeepRacer总决赛冠军杯也证明了这一点。
英文摘要：
Deep reinforcement learning has achieved significant results in low-level controlling tasks. However, for some applications like autonomous driving and drone flying, it is difficult to control behavior stably since the agent may suddenly change its actions which often lowers the controlling system's efficiency, induces excessive mechanical wear, and causes uncontrollable, dangerous behavior to the vehicle. Recently, a method called conditioning for action policy smoothness (CAPS) was proposed to solve the problem of jerkiness in low-dimensional features for applications such as quadrotor drones. To cope with high-dimensional features, this paper proposes image-based regularization for action smoothness (I-RAS) for solving jerky control in autonomous miniature car racing. We also introduce a control based on impact ratio, an adaptive regularization weight to control the smoothness constraint, called IR control. In the experiment, an agent with I-RAS and IR control significantly improves the success rate from 59% to 95%. In the real-world-track experiment, the agent also outperforms other methods, namely reducing the average finish lap time, while also improving the completion rate even without real world training. This is also justified by an agent based on I-RAS winning the 2022 AWS DeepRacer Final Championship Cup.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
