Diffusion-ES：用于自动驾驶和零样本指令跟随的基于扩散的无梯度规划
Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following
AI总结：
Diffusion-ES将无梯度进化搜索与扩散模型去噪结合，优化黑盒非可微目标，在自动驾驶基准nuPlan上达到最先进性能，并能通过少样本LLM提示优化语言奖励，实现零样本指令跟随和复杂行为生成。
摘要：
扩散模型在建模决策和控制中复杂且多模态的轨迹分布方面表现出色。最近提出了奖励梯度引导的去噪方法，以生成既最大化可微奖励函数又最大化扩散模型捕获的数据分布下似然的轨迹。奖励梯度引导的去噪需要拟合干净样本和噪声样本的可微奖励函数，这限制了其作为通用轨迹优化器的适用性。在本文中，我们提出了DiffusionES，一种将无梯度优化与轨迹去噪相结合的方法，以优化黑盒非可微目标，同时保持数据流形。Diffusion-ES在进化搜索期间从扩散模型采样轨迹，并使用黑盒奖励函数对其进行评分。它使用截断的扩散过程对高得分轨迹进行变异，该过程应用少量噪声和去噪步骤，从而允许更高效地探索解空间。我们表明，DiffusionES在nuPlan（一个成熟的自动驾驶闭环规划基准）上实现了最先进的性能。Diffusion-ES优于现有的基于采样的规划器、反应式确定性或基于扩散的策略以及奖励梯度引导。此外，我们表明，与先前的引导方法不同，我们的方法可以优化由少样本LLM提示生成的非可微语言形状奖励函数。当由发出指令跟随的人类教师引导时，我们的方法可以生成新颖且高度复杂的行为，例如训练数据中不存在的激进车道穿插。这使我们能够解决现有轨迹优化方法和驾驶策略无法胜任的最困难的nuPlan场景。
英文摘要：
Diffusion models excel at modeling complex and multimodal trajectory distributions for decision-making and control. Reward-gradient guided denoising has been recently proposed to generate trajectories that maximize both a differentiable reward function and the likelihood under the data distribution captured by a diffusion model. Reward-gradient guided denoising requires a differentiable reward function fitted to both clean and noised samples, limiting its applicability as a general trajectory optimizer. In this paper, we propose DiffusionES, a method that combines gradient-free optimization with trajectory denoising to optimize black-box non-differentiable objectives while staying in the data manifold. Diffusion-ES samples trajectories during evolutionary search from a diffusion model and scores them using a black-box reward function. It mutates high-scoring trajectories using a truncated diffusion process that applies a small number of noising and denoising steps, allowing for much more efficient exploration of the solution space. We show that DiffusionES achieves state-of-the-art performance on nuPlan, an established closed-loop planning benchmark for autonomous driving. Diffusion-ES outperforms existing sampling-based planners, reactive deterministic or diffusion-based policies, and reward-gradient guidance. Additionally, we show that unlike prior guidance methods, our method can optimize non-differentiable language-shaped reward functions generated by few-shot LLM prompting. When guided by a human teacher that issues instructions to follow, our method can generate novel, highly complex behaviors, such as aggressive lane weaving, which are not present in the training data. This allows us to solve the hardest nuPlan scenarios which are beyond the capabilities of existing trajectory optimization methods and driving policies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
