基于最大离散竞争的语义分割失败暴露
Exposing Semantic Segmentation Failures via Maximum Discrepancy Competition
摘要：
语义分割是计算机视觉中一个被广泛研究的课题，每年都有许多方法被提出。由于深度学习在语义分割中的出现，现有基准测试的性能接近饱和。一个自然的问题随之而来：封闭的（和频繁重用的）测试集上的优越性能是否转移到具有不受约束变化的开放视觉世界？在本文中，我们采取措施回答这个问题，暴露现有的语义分割方法在开放的视觉世界的限制下，非常有限的人类标记的努力失败。受以往模型伪造研究的启发，我们从一个任意大的图像集开始，通过最大化两种分割方法之间的离散度（MAD）来自动采样一个小的图像集。所选图像最有可能伪造这两种方法中的一种（或两种）。我们还明确实施了几个条件，以多样化暴露的故障，对应于不同的根本原因。在MAD竞争中，失败更难暴露的分割方法被认为是更好的。我们对10个PASCAL VOC语义分割算法进行了彻底的MAD诊断。通过对实验结果的详细分析，指出了竞争算法的优势和不足，以及语义分割进一步发展的潜在研究方向。这些代码可在\url{https://github.com/QTJiebin/MAD_Segmentation}上公开获取。
英文摘要：
Semantic segmentation is an extensively studied task in computer vision, with numerous methods proposed every year. Thanks to the advent of deep learning in semantic segmentation, the performance on existing benchmarks is close to saturation. A natural question then arises: Does the superior performance on the closed (and frequently re-used) test sets transfer to the open visual world with unconstrained variations? In this paper, we take steps toward answering the question by exposing failures of existing semantic segmentation methods in the open visual world under the constraint of very limited human labeling effort. Inspired by previous research on model falsification, we start from an arbitrarily large image set, and automatically sample a small image set by MAximizing the Discrepancy (MAD) between two segmentation methods. The selected images have the greatest potential in falsifying either (or both) of the two methods. We also explicitly enforce several conditions to diversify the exposed failures, corresponding to different underlying root causes. A segmentation method, whose failures are more difficult to be exposed in the MAD competition, is considered better. We conduct a thorough MAD diagnosis of ten PASCAL VOC semantic segmentation algorithms. With detailed analysis of experimental results, we point out strengths and weaknesses of the competing algorithms, as well as potential research directions for further advancement in semantic segmentation. The codes are publicly available at \url{https://github.com/QTJiebin/MAD_Segmentation}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
