Cityscape-Adverse：通过基于扩散的图像编辑实现真实场景修改的语义分割鲁棒性基准测试
Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation with Realistic Scene Modifications via Diffusion-Based Image Editing
AI总结：
该研究提出Cityscape-Adverse基准，用扩散图像编辑模拟八种不利场景条件，测试主流语义分割模型鲁棒性，发现Transformer模型比CNN更具恢复力，且在该基准上训练可提升模型跨域鲁棒性。
摘要：
生成式人工智能的最新进展，尤其是基于扩散的图像编辑技术，已能仅通过文本指令将图像转换为高度真实的场景。该技术为生成多样化合成数据集以评估模型鲁棒性提供了巨大潜力。本文提出Cityscape-Adverse基准，该基准采用基于扩散的图像编辑模拟八种不利条件，包括天气、光照和季节变化，同时保留原始语义标签。我们评估了基于扩散的模型生成真实场景修改的可靠性，并测试了当前最先进的基于CNN和Transformer的语义分割模型在这些挑战性条件下的性能。此外，我们分析了哪些修改对模型性能影响最大，并探索了在合成数据集上训练如何提升真实世界不利场景下的鲁棒性。结果表明，所有测试模型，尤其是基于CNN的架构，在极端条件下均出现显著性能下降，而基于Transformer的模型表现出更强的恢复力。我们验证了在Cityscape-Adverse上训练的模型应用于未见过的域时，恢复力显著提升。代码和数据集将发布于https://github.com/naufalso/cityscape-adverse。
英文摘要：
Recent advancements in generative AI, particularly diffusion-based image editing, have enabled the transformation of images into highly realistic scenes using only text instructions. This technology offers significant potential for generating diverse synthetic datasets to evaluate model robustness. In this paper, we introduce Cityscape-Adverse, a benchmark that employs diffusion-based image editing to simulate eight adverse conditions, including variations in weather, lighting, and seasons, while preserving the original semantic labels. We evaluate the reliability of diffusion-based models in generating realistic scene modifications and assess the performance of state-of-the-art CNN and Transformer-based semantic segmentation models under these challenging conditions. Additionally, we analyze which modifications have the greatest impact on model performance and explore how training on synthetic datasets can improve robustness in real-world adverse scenarios. Our results demonstrate that all tested models, particularly CNN-based architectures, experienced significant performance degradation under extreme conditions, while Transformer-based models exhibited greater resilience. We verify that models trained on Cityscape-Adverse show significantly enhanced resilience when applied to unseen domains. Code and datasets will be released at https://github.com/naufalso/cityscape-adverse.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
