FROST-Drive: 可扩展且高效的端到端驾驶方法，采用冻结的视觉编码器
FROST-Drive: Scalable and Efficient End-to-End Driving with a Frozen Vision Encoder
AI总结：
FROST-Drive通过冻结预训练视觉编码器，结合适配器和解码器，实现高效端到端驾驶，优于全微调方法。
摘要：
端到端（E2E）模型在自动驾驶中旨在直接将传感器输入映射到控制命令，但其在新型和复杂场景中的泛化能力仍是一个关键挑战。传统上，对驾驶数据集上的视觉编码器进行完全微调可能会限制其泛化能力，因为模型会过于专门化于训练数据。本文挑战这种训练范式。我们提出FROST-Drive，一种新的E2E架构，旨在保留和利用来自视觉语言模型（VLM）的预训练视觉编码器的强大泛化能力。通过保持编码器的权重冻结，我们的方法直接将VLM中的丰富、泛化世界知识转移到驾驶任务。我们的模型架构结合了这个冻结的编码器与基于Transformer的适配器进行多模态融合，以及基于GRU的解码器用于平滑的路径点生成。此外，我们引入了一个定制的损失函数，旨在直接优化Rater Feedback Score（RFS），这是一个优先考虑稳健轨迹规划的指标。我们对Waymo Open E2E数据集进行了广泛的实验，这是一个大规模数据集，特意编纂以捕捉长尾场景，证明我们的冻结编码器方法显著优于采用完全微调的模型。我们的结果提供了有力的证据，表明保留一个有能力VLM的广泛知识比进行密集的领域特定适应更有效，以实现稳健、可泛化的驾驶性能。这为开发能够更好地处理现实世界应用领域复杂性的基于视觉的模型开辟了新路径。
英文摘要：
End-to-end (E2E) models in autonomous driving aim to directly map sensor inputs to control commands, but their ability to generalize to novel and complex scenarios remains a key challenge. The common practice of fully fine-tuning the vision encoder on driving datasets potentially limits its generalization by causing the model to specialize too heavily in the training data. This work challenges the necessity of this training paradigm. We propose FROST-Drive, a novel E2E architecture designed to preserve and leverage the powerful generalization capabilities of a pretrained vision encoder from a Vision-Language Model (VLM). By keeping the encoder's weights frozen, our approach directly transfers the rich, generalized world knowledge from the VLM to the driving task. Our model architecture combines this frozen encoder with a transformer-based adapter for multimodal fusion and a GRU-based decoder for smooth waypoint generation. Furthermore, we introduce a custom loss function designed to directly optimize for Rater Feedback Score (RFS), a metric that prioritizes robust trajectory planning. We conduct extensive experiments on Waymo Open E2E Dataset, a large-scale datasets deliberately curated to capture the long-tail scenarios, demonstrating that our frozen-encoder approach significantly outperforms models that employ full fine-tuning. Our results provide substantial evidence that preserving the broad knowledge of a capable VLM is a more effective strategy for achieving robust, generalizable driving performance than intensive domain-specific adaptation. This offers a new pathway for developing vision-based models that can better handle the complexities of real-world application domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
