发表机构
University of California, Los Angeles; Toyota Research Institute（加州大学洛杉矶分校; 丰田研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
DreamStream提出策略导向的生成式闭环仿真器，通过自回归视频模型和FDπ指标缩小仿真到真实差距，并构建Navhard-CL基准以暴露驾驶策略失败模式。
AI 中文摘要
在仿真中忠实评估端到端驾驶策略，需要不仅具有照片级真实感、而且保留策略决策所依赖的场景特征的观测。然而，现有平台存在仿真到真实的视觉差距，这会破坏策略的感知，削弱其评估策略闭环决策的能力。为此，我们提出DreamStream，一种生成式闭环仿真器，通过基于仿真器锚定的自回归视频模型实现策略导向的保真度。我们的视频模型通过交通布局引导从大型预训练视频模型中蒸馏而来，在变化视觉外观的同时保留策略相关特征，如场景布局和动态物体的时间一致性。我们进一步观察到，诸如FID之类的感知指标会错误排序这些特征的保留程度。为解决此问题，我们引入FDπ，一种新的多表示指标，将仿真到真实的差距度量为其在公开端到端策略的场景上下文特征上的Fréchet距离。在FDπ下，DreamStream在nuScenes上比最强先前的闭环仿真器提升1.6倍，在NAVSIM上提升4.7倍，并对策略的感知可观测性诱导最小扰动。基于DreamStream，我们构建了Navhard-CL基准，将非反应性的真实世界基准NAVSIM转变为具有对抗性驾驶行为和天气变化的交互式测试环境。该基准暴露了驾驶策略的许多失败模式，如评分器偏差和缺乏恢复行为，这些是先前闭环基准所忽视的。代码和数据可在该https URL获取。
英文摘要
Faithfully evaluating end-to-end driving policies in simulation requires observations that are not merely photo-realistic, but preserve the scene features a policy relies on to make decisions. Existing platforms, however, exhibit a sim-to-real visual gap that corrupts policy perception, undermining their ability to assess a policy's closed-loop decision-making. To this end, we propose DreamStream, a generative, closed-loop simulator that achieves policy-oriented fidelity using a simulator-grounded autoregressive video model. Our video model is distilled from a large pretrained video model via traffic layout guidance, varying visual appearance while preserving policy-relevant features such as scenario layout and the temporal consistency of dynamic objects. We further observe that perceptual metrics like FID misrank how well these features are preserved. To tackle this, we introduce FD$π$, a new multi-representation metric that measures the sim-to-real gap as the Fréchet distance over scene-context features from public E2E policies. Under FD$π$, DreamStream improves over the strongest prior closed-loop simulator by $1.6\times$ on nuScenes and $4.7\times$ on NAVSIM, and induces the least perturbation to policy's perceptual observability. Based on DreamStream, we construct Navhard-CL benchmark, which turns non-reactive real-world benchmark NAVSIM into interactive testing environments with adversarial driving behaviors and weather variations. This benchmark exposes many failure modes of driving policies, such as scorer bias and lack of recovery behaviors, that prior closed-loop benchmarks overlook. Code and data are available at https://github.com/VAIL-UCLA/DreamStream.
CommentsAccepted to CoRL 2026. Project page: https://vail-ucla.github.io/DreamStream/
