CRISP：一种通过基于预测的世界模型预训练实现驾驶的时空相机-雷达主干网络
CRISP: A Spatiotemporal Camera-Radar Backbone for Driving via Forecasting-Based World-Model Pretraining
AI总结：
研究通过基于预测的表示学习预训练时空相机-雷达主干网络CRISP，利用历史多视图图像和雷达扫描，通过预测未来激光雷达点云学习统一鸟瞰图表示，介绍相关组件，实验表明其能提升点云预测并有效迁移至下游任务。
摘要：
相机-雷达（CR）融合是自动驾驶的实用传感配置，但现有模型通常用特定任务监督训练，限制了可复用表示学习。我们提出CRISP，一种通过基于预测的表示学习预训练的时空CR主干网络。给定历史多视图图像和雷达扫描，CRISP通过预测未来LiDAR点云学习统一的鸟瞰图（BEV）表示。LiDAR仅在预训练期间用作特权监督；部署模型仅需相机和雷达。为使基于预测的预训练对CR融合有效，CRISP引入增强雷达编码器、雷达增强的时间自注意力和具有模态创新门控的多模态特征渲染。这些组件将雷达距离和多普勒线索注入BEV时间传播，并允许BEV令牌选择性地纳入相机和雷达证据。在nuScenes上的实验表明，CRISP改进了长距离点云预测，并有效地转移到下游任务，包括3D检测、跟踪、在线映射、运动预测、未来占用预测和规划，表明预测性CR预训练是在实际传感器配置下实现可扩展驾驶表示的一条有前途的途径。项目网站是这个https URL。
英文摘要：
Camera-radar (CR) fusion is a practical sensing configuration for autonomous driving, but existing models are typically trained with task-specific supervision, limiting reusable representation learning. We present CRISP, a spatiotemporal CR backbone pretrained through forecasting-based representation learning. Given historical multi-view images and radar sweeps, CRISP learns a unified bird's-eye-view (BEV) representation by predicting future LiDAR point clouds. LiDAR is used only as privileged supervision during pretraining; the deployed model requires only camera and radar. To make forecasting-based pretraining effective for CR fusion, CRISP introduces an enhanced radar encoder, radar-enhanced temporal self-attention, and multimodal feature rendering with modality innovation gating. These components inject radar range and Doppler cues into BEV temporal propagation and allow BEV tokens to selectively incorporate camera and radar evidence. Experiments on nuScenes show that CRISP improves long-horizon point cloud forecasting and transfers effectively to downstream tasks, including 3D detection, tracking, online mapping, motion forecasting, future occupancy prediction, and planning, suggesting that predictive CR pretraining is a promising path toward scalable driving representations under practical sensor configurations. The project website is https://umfieldrobotics.github.io/CRISP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
