恶劣天气下自动驾驶的真实世界感知：通过基础模型引导的自动标注增强标准检测器

中文摘要 AI 辅助

标准部署就绪的自动驾驶目标检测器在恶劣天气和光照条件下性能会下降，除非在大量特定领域数据上进行训练。虽然大规模视觉基础模型提供了强大的零样本泛化能力，但其高计算成本使其不适合实时部署。为弥合这一差距，我们提出了一种基础模型引导的自动标注流水线，无需架构更改即可增强标准检测器。我们首先在我们自定义的真实世界驾驶数据集上对三种不同模型（YOLOv8、Co-DETR 和 SAM3）进行基准测试，该数据集涵盖跨不同路线、天气和光照条件的 25 种独特运行场景。基于我们的分析，SAM3 在所有场景中表现出卓越的准确性和鲁棒性。因此，我们将其部署为离线自动标注器，为数据集中未标注的子集生成伪标签。在这些标注上微调基线 YOLOv8，与基线模型相比，总体平均精度（mAP）提高了 16.04%，并改善了跨环境稳定性，其中在住宅区直射阳光和高速公路雾天场景中，mAP 分别提高了 32.73% 和 28.65%。这些结果表明，标准检测器无需大量手动标注或架构修改即可实现环境鲁棒性。

英文摘要

Standard deployment-ready object detectors for autonomous vehicles degrade in adverse weather and lighting conditions without being trained on extensive domain-specific data. While large-scale vision foundation models offer robust zero-shot generalization, their high computational cost makes them impractical for real-time deployment. To bridge this gap, we propose a foundation-guided auto-annotation pipeline that enhances standard detectors without architectural changes. We first benchmark three distinct models, YOLOv8, Co-DETR, and SAM3, on our custom real-world driving dataset spanning 25 unique operational scenarios across various route, weather, and lighting conditions. Based on our analysis, SAM3 demonstrates superior accuracy and resilience across all scenarios. Thus, we deploy it as an offline auto-annotator to generate pseudo-labels on the unannotated subset of our dataset. Fine-tuning the baseline YOLOv8 on these annotations yields a 16.04% higher overall mean Average Precision (mAP) and improves cross-environmental stability compared to the baseline model, highlighted by a 32.73% and 28.65% mAP increase in Residential Direct Sunlight and Highway Fog, respectively. These results demonstrate that standard detectors can achieve environmental resilience without the need for extensive manual annotation or architectural modifications.

发表机构

• Virginia Commonwealth University (VCU)（弗吉尼亚联邦大学）

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

Submitted to IEEE for review