发表机构
University of Copenhagen; Royal Danish Academy（哥本哈根大学; 丹麦皇家科学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对图像与标签不对齐影响分割模型性能的问题，提出AnS方法，基于空间变换器模块，利用仿射变换转换标签，通过自监督正则化损失防止捷径学习，无需黄金标签，可同时实现高质量建筑物分割与精确标签-图像对齐。
AI 中文摘要
图像分割的监督学习通常需要空间对齐的图像和标签集。当图像和标签来自不同来源时，配对可能会不对齐，这会显著降低学习模型的性能，在遥感领域尤为常见。本文提出一种针对不对齐标签进行训练的新方法，同时学习标签对齐。我们的对齐与分割（AnS）方法基于空间变换器模块，使用仿射变换来转换不对齐的标签，为规范语义分割网络提供更好的学习目标。通过自监督正则化损失防止语义分割网络中不对齐标签的捷径学习，且与数据增强互补。AnS方法无需任何黄金标签进行学习。实验表明，该方法能在不同城市的合成数据和真实数据上同时实现高质量的建筑物分割和精确的标签-图像对齐。
英文摘要
Supervised learning for image segmentation typically requires spatially aligned image and label sets. When images and labels originate from different sources, the pairing may be misaligned, which can significantly deteriorate the performance of the learned models. This is especially common in remote sensing, when aerial or satellite images are co-registered with labels from another source (e.g., OpenStreetMap). In this work, we propose a novel approach for training on misaligned labels, where we simultaneously learn the label alignment. Our align and segment (AnS) approach builds on the spatial transformer module to transform the misaligned labels using an affine transformation to provide a better learning target for a canonical semantic segmentation network. We prevent shortcut learning of misaligned labels in these semantic segmentation networks through a self-supervised regularization loss and show that it is complementary to data augmentation, especially for systematically misaligned training data. A decisive characteristic of our AnS approach is that it learns without requiring any golden labels. We experimentally show on both synthetic and real-world data from different cities that our approach enables high-quality building segmentation and precise label-image alignment at the same time. Code and derived datasets are available at https://github.com/venkanna37/align-and-segment
Commentsmain draft with references is 17 pages
