arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.14737cs.RO
收藏

跨多种恶劣天气条件的可泛化自动驾驶系统

Generalizable Autonomous Driving System across Diverse Adverse Weather Conditions

The University of Hong Kong(香港大学);Southern University of Science and Technology(南方科技大学);Shenzhen Research Institute of Big Data(深圳大数据研究院)

Wei-Bin Kou, Guangxu Zhu, Rongguang Ye, Qingfeng Lin, Zeyi Ren, Ming Tang, Yik-Chung Wu

AI总结:

针对自动驾驶在混合恶劣天气下的分割难题,提出无参考的 AdvImmu 框架,利用短时天气不变性并结合 SAM 伪标签,显著提升跨天气泛化性能。

摘要:

各种恶劣天气条件对自动驾驶(AD)街景语义理解(分割)构成了重大挑战。一种常见策略是最小化晴朗与恶劣天气条件下拍摄图像之间的差异。然而,该技术通常依赖于使用晴朗图像作为参考,而这在实践中很难获得。此外,这种方法通常只针对单一恶劣条件,因此在面对多种恶劣天气条件混合时表现不佳。为解决这些问题,我们提出了一种无参考且对恶劣天气免疫的方案(称为 AdvImmu),该方案利用短时间内(数秒)天气条件的不变性。具体而言,AdvImmu 包含三个组成部分:局部时序机制(LSM)、全局混洗机制(GSM)和展开正则化器(URs)。LSM 利用相邻帧之间的时间相关性来增强模型性能。GSM 被提出用于混洗 LSM 片段,以防止对时间模式的过拟合。URs 是所提出的两个正则化器的深度展开实现,用于惩罚模型复杂性,以增强跨天气泛化能力。此外,为克服 AdvImmu 训练中对连续逐帧标注的过度依赖(这在 AD 场景中通常不可用),我们引入了名为 Segment Anything Model(SAM)的基础模型来辅助标注帧,并额外提出了一种聚类算法(记为 SBICAC)来克服 SAM 的类别无关问题,以生成伪标签。大量实验表明,所提出的 AdvImmu 在平均交并比(mIoU)上比现有最先进方法高出 88.56%。

英文摘要:

Various adverse weather conditions pose a significant challenge to autonomous driving (AD) street scene semantic understanding (segmentation). A common strategy is to minimize the disparity between images captured in clear and adverse weather conditions. However, this technique typically relies on utilizing clear image as a reference, which is challenging to obtain in practice. Furthermore, this method typically targets a single adverse condition, and thus perform poorly when confronting a mixture of multiple adverse weather conditions. To address these issues, we introduce a reference-free and Adverse weather-Immune scheme (called AdvImmu) that leverages the invariance of weather conditions over short periods (seconds). Specifically, AdvImmu includes three components: Locally Sequential Mechanism (LSM), Globally Shuffled Mechanism (GSM), and Unfolded Regularizers (URs). LSM leverages temporal correlations between adjacent frames to enhance model performance. GSM is proposed to shuffle LSM segments to prevent overfitting of temporal patterns. URs are the deep unfolding implementation of two proposed regularizers to penalize the model complexity to enhance across-weather generalization. In addition, to overcome the over-reliance on consecutive frame-wise annotations in the training of AdvImmu (typically unavailable in AD scenarios), we incorporate a foundation model named Segment Anything Model (SAM) to assist to annotate frames, and additionally propose a cluster algorithm (denoted as SBICAC) to surmount SAM's category-agnostic issue to generate pseudo-labels. Extensive experiments demonstrate that the proposed AdvImmu outperforms existing state-of-the-art methods by 88.56% in mean Intersection over Union (mIoU).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑