arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.09231cs.CVcs.LG
收藏

语义分割中的可靠性:我们能使用合成数据吗?

Reliability in Semantic Segmentation: Can We Use Synthetic Data?

LIGM, Ecole des Ponts, Univ Gustave Eiffel, CNRS(LIGM、巴黎路桥学校、古斯塔夫·埃菲尔大学、法国国家科学研究中心);kyutai(Kyutai公司);Sorbonne Université(索邦大学)

Thibaut Loiseau, Tuan-Hung Vu, Mickael Chen, Patrick Pérez, Matthieu Cord

AI总结:

本文首次提出利用域内数据微调Stable Diffusion生成合成OOD数据,以全面评估语义分割模型在现实世界中的鲁棒性,并证明其与真实OOD数据评估结果高度相关,同时可增强模型校准与OOD检测能力。

摘要:

评估感知模型对协变量偏移的鲁棒性及其检测分布外(OOD)输入的能力,对于自动驾驶等安全关键应用至关重要。然而,由于此类应用的本质,相关数据难以收集和标注。本文首次展示了如何专门生成合成数据,以全面评估语义分割模型在现实世界中的可靠性。通过仅使用域内数据微调Stable Diffusion,我们在OOD领域执行零样本生成视觉场景,或用OOD对象进行图像修复。该合成数据被用于评估预训练分割器的鲁棒性,从而为它们在遇到真实边缘案例时的性能提供见解。通过大量实验,我们证明了模型在我们的合成OOD数据上评估时的性能与在真实OOD输入上评估时的性能之间存在高度相关性,表明了这种虚拟测试的相关性。此外,我们展示了如何利用我们的方法来增强分割器的校准和OOD检测能力。代码和数据已公开。

英文摘要:

Assessing the robustness of perception models to covariate shifts and their ability to detect out-of-distribution (OOD) inputs is crucial for safety-critical applications such as autonomous vehicles. By nature of such applications, however, the relevant data is difficult to collect and annotate. In this paper, we show for the first time how synthetic data can be specifically generated to assess comprehensively the real-world reliability of semantic segmentation models. By fine-tuning Stable Diffusion with only in-domain data, we perform zero-shot generation of visual scenes in OOD domains or inpainted with OOD objects. This synthetic data is employed to evaluate the robustness of pretrained segmenters, thereby offering insights into their performance when confronted with real edge cases. Through extensive experiments, we demonstrate a high correlation between the performance of models when evaluated on our synthetic OOD data and when evaluated on real OOD inputs, showing the relevance of such virtual testing. Furthermore, we demonstrate how our approach can be utilized to enhance the calibration and OOD detection capabilities of segmenters. Code and data are made public.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑