桥接视角：用于3D目标检测与跟踪的基础模型引导BEV地图
Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Tracking
AI总结：
提出DualViewDistill混合框架，通过DINOv2蒸馏生成基础模型引导的BEV地图，结合PV特征与可变形聚合，在nuScenes和Argoverse 2上实现SOTA的3D检测与跟踪。
摘要：
基于摄像头的3D目标检测与跟踪对于自动驾驶的感知至关重要。当前最先进的方法通常仅依赖透视视图（PV）或鸟瞰图（BEV）特征，限制了其同时利用细粒度物体细节和空间结构化场景表示的能力。在本研究中，我们提出了DualViewDistill，这是一种混合检测与跟踪框架，结合了PV和BEV相机图像特征，以利用它们互补的优势。我们的方法引入了由基础模型引导的BEV地图，利用描述性的DINOv2特征，通过一种新颖的蒸馏过程将其蒸馏到BEV表示中。通过将PV特征与富含DINOv2语义和几何特征的BEV地图相结合，我们的模型通过可变形聚合利用这种混合表示，以增强3D目标检测与跟踪。在nuScenes和Argoverse 2基准上的大量实验表明，DualViewDistill实现了最先进的性能。结果展示了基础模型BEV地图在实现更可靠自动驾驶感知方面的潜力。我们在 https://dualviewdistill.cs.uni-freiburg.de 提供了代码和预训练模型。
英文摘要：
Camera-based 3D object detection and tracking are essential for perception in autonomous driving. Current state-of-the-art approaches often rely exclusively on either perspective-view (PV) or bird's-eye-view (BEV) features, limiting their ability to leverage both fine-grained object details and spatially structured scene representations. In this work, we propose DualViewDistill, a hybrid detection and tracking framework that incorporates both PV and BEV camera image features to leverage their complementary strengths. Our approach introduces BEV maps guided by foundation models, leveraging descriptive DINOv2 features that are distilled into BEV representations through a novel distillation process. By integrating PV features with BEV maps enriched with semantic and geometric features from DINOv2, our model leverages this hybrid representation via deformable aggregation to enhance 3D object detection and tracking. Extensive experiments on the nuScenes and Argoverse 2 benchmarks demonstrate that DualViewDistill achieves state-of-the-art performance. The results showcase the potential of foundation model BEV maps to enable more reliable perception for autonomous driving. We make the code and pre-trained models available at https://dualviewdistill.cs.uni-freiburg.de .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
