PEM：基于原型的图像分割高效MaskFormer
PEM: Prototype-based Efficient MaskFormer for Image Segmentation
AI总结：
本文提出高效Transformer架构PEM，通过基于原型的交叉注意力和高效多尺度特征金字塔网络，在语义与全景分割任务上以更低计算成本取得优于任务专用架构的性能。
摘要：
近年来，基于Transformer的架构在图像分割领域取得了令人瞩目的成果。凭借其灵活性，这些架构在单一统一框架下于语义分割和全景分割等多个分割任务中获得了卓越的性能。然而，为了实现这种出色表现，这些架构采用了密集运算并需要大量计算资源，而这在边缘设备上往往难以满足。为填补这一空白，我们提出了基于原型的高效MaskFormer（PEM），一种可在多个分割任务中运行的高效Transformer架构。PEM提出了一种新颖的基于原型的交叉注意力机制，利用视觉特征的冗余来限制计算量，在不损害性能的前提下提高效率。此外，PEM引入了一种高效的多尺度特征金字塔网络，通过结合可变形卷积和基于上下文的自我调制，能够以高效方式提取具有丰富语义内容的特征。我们在语义分割和全景分割两个任务上对所提出的PEM架构进行了基准测试，并在Cityscapes和ADE20K两个不同数据集上进行了评估。PEM在每个任务和数据集上均展现出卓越的性能，超越了任务专用架构，同时与计算开销高昂的基线方法相比具有可比性甚至更优。
英文摘要：
Recent transformer-based architectures have shown impressive results in the field of image segmentation. Thanks to their flexibility, they obtain outstanding performance in multiple segmentation tasks, such as semantic and panoptic, under a single unified framework. To achieve such impressive performance, these architectures employ intensive operations and require substantial computational resources, which are often not available, especially on edge devices. To fill this gap, we propose Prototype-based Efficient MaskFormer (PEM), an efficient transformer-based architecture that can operate in multiple segmentation tasks. PEM proposes a novel prototype-based cross-attention which leverages the redundancy of visual features to restrict the computation and improve the efficiency without harming the performance. In addition, PEM introduces an efficient multi-scale feature pyramid network, capable of extracting features that have high semantic content in an efficient way, thanks to the combination of deformable convolutions and context-based self-modulation. We benchmark the proposed PEM architecture on two tasks, semantic and panoptic segmentation, evaluated on two different datasets, Cityscapes and ADE20K. PEM demonstrates outstanding performance on every task and dataset, outperforming task-specific architectures while being comparable and even better than computationally-expensive baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
