非平稳环境下基于在线预测自适应的自适应驾驶
Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptation
摘要:
在深度表示学习的支持下,强化学习(RL)提供了一个端到端的学习框架,能够解决自动驾驶(SD)任务,而无需手动设计。然而,时变的非平稳环境会导致熟练但专业的RL策略在执行时失败。例如,在晴天训练的基于RL的SD策略不能很好地推广到雨天。尽管Meta学习使RL代理能够适应新的任务/环境,但其离线操作无法使代理在面对非平稳环境时具有在线适应能力。提出了一种基于\emph{conjectural online lookahead adaptation}(COLA)的在线Meta强化学习算法。COLA通过最大化代理对前瞻视野中未来性能的推测来确定每一步的在线适应。实验结果表明,在动态变化的天气和光照条件下,基于COLA的自适应驾驶在在线适应性方面优于基线策略。演示视频、源代码和附录可从{ \tt https://github.com/Panshark/COLA}获得
英文摘要:
Powered by deep representation learning, reinforcement learning (RL) provides an end-to-end learning framework capable of solving self-driving (SD) tasks without manual designs. However, time-varying nonstationary environments cause proficient but specialized RL policies to fail at execution time. For example, an RL-based SD policy trained under sunny days does not generalize well to rainy weather. Even though meta learning enables the RL agent to adapt to new tasks/environments, its offline operation fails to equip the agent with online adaptation ability when facing nonstationary environments. This work proposes an online meta reinforcement learning algorithm based on the \emph{conjectural online lookahead adaptation} (COLA). COLA determines the online adaptation at every step by maximizing the agent's conjecture of the future performance in a lookahead horizon. Experimental results demonstrate that under dynamically changing weather and lighting conditions, the COLA-based self-adaptive driving outperforms the baseline policies in terms of online adaptability. A demo video, source code, and appendixes are available at {\tt https://github.com/Panshark/COLA}
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。