arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.16397cs.LGcs.AI
收藏

面向随机驾驶环境的不确定性感知决策Transformer

Uncertainty-Aware Decision Transformer for Stochastic Driving Environments

Tsinghua(清华大学);Shanghai Qi Zhi(上海期智);Stanford(斯坦福大学);QCraft(轻舟智航)

Zenan Li, Fan Nie, Qiao Sun, Fang Da, Hang Zhao

AI总结:

针对随机驾驶环境下传统序列建模式离线RL失效的问题,提出基于条件互信息估计不确定性、采用截断回报学习的UNREST模型,实现谨慎规划并提升驾驶场景性能。

摘要:

离线强化学习(Offline RL)无需主动交互即可完成策略学习,因此尤其适用于自动驾驶任务。近期Transformer的成功启发研究者将离线强化学习建模为序列建模问题,但该方法在随机环境中会失效,因其错误假设相同动作总能达成相同目标。本文提出一种不确定性感知决策Transformer(UNcertainty-awaRE deciSion Transformer,UNREST),用于随机驾驶环境下的规划,无需引入额外的转移模型或复杂生成模型。具体而言,UNREST通过转移与回报之间的条件互信息估计不确定性。我们发现驾驶环境存在“不确定性累积”和“时间局部性”特性,因此将决策Transformer中的全局回报替换为受环境影响更小的截断回报,从而从动作的实际结果而非环境转移中学习。我们还在推理阶段动态评估不确定性以实现谨慎规划。大量实验证明,UNREST在多种驾驶场景中表现优异,同时验证了我们的不确定性估计策略的有效性。

英文摘要:

Offline Reinforcement Learning (RL) enables policy learning without active interactions, making it especially appealing for self-driving tasks. Recent successes of Transformers inspire casting offline RL as sequence modeling, which, however, fails in stochastic environments with incorrect assumptions that identical actions can consistently achieve the same goal. In this paper, we introduce an UNcertainty-awaRE deciSion Transformer (UNREST) for planning in stochastic driving environments without introducing additional transition or complex generative models. Specifically, UNREST estimates uncertainties by conditional mutual information between transitions and returns. Discovering 'uncertainty accumulation' and 'temporal locality' properties of driving environments, we replace the global returns in decision transformers with truncated returns less affected by environments to learn from actual outcomes of actions rather than environment transitions. We also dynamically evaluate uncertainty at inference for cautious planning. Extensive experiments demonstrate UNREST's superior performance in various driving scenarios and the power of our uncertainty estimation strategy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑