arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.14043cs.CV
收藏

基于早期区域代理的语义分割

Semantic Segmentation by Early Region Proxy

Shanghai Jiao Tong University(上海交通大学);Qing Yuan Research Institute(清源研究院);AI Institute, Shanghai Jiao Tong University(上海交通大学人工智能研究院);Shanghai Qi Zhi Institute(上海期智研究院)

Yifan Zhang, Bo Pang, Cewu Lu

摘要:

典型的视觉骨干处理结构化特征。作为一种折衷,语义分割长期以来一直被建模为密集规则网格上的逐点预测。在这项工作中,我们提出了一种新的和有效的建模,从解释图像作为一个可学习的区域,其中每个区域具有灵活的几何形状,并进行同质语义的镶嵌。为了对区域上下文进行建模,我们利用Transformer通过在区域嵌入上应用多层自关注来以序列到序列的方式对区域进行编码,区域嵌入作为特定区域的代理。语义分割现在使用单个线性分类器在编码区域嵌入之上作为每区域预测来执行,其中不再需要解码器。建议RegProxy模型放弃了常见的笛卡尔特征布局,纯粹在区域级别上操作。因此,它表现出最有竞争力的性能效率权衡与传统的密集预测方法相比。例如,在ADE 20 K上,小型RegProxy-S/16的性能优于最好的CNN模型,使用25%的参数和4%的计算,而最大的RegProxy-L/16实现了52.9mIoU,比最先进的模型高出2.1%,资源更少。代码和模型可在https://github.com/YiF-Zhang/RegionProxy上获得。

英文摘要:

Typical vision backbones manipulate structured features. As a compromise, semantic segmentation has long been modeled as per-point prediction on dense regular grids. In this work, we present a novel and efficient modeling that starts from interpreting the image as a tessellation of learnable regions, each of which has flexible geometrics and carries homogeneous semantics. To model region-wise context, we exploit Transformer to encode regions in a sequence-to-sequence manner by applying multi-layer self-attention on the region embeddings, which serve as proxies of specific regions. Semantic segmentation is now carried out as per-region prediction on top of the encoded region embeddings using a single linear classifier, where a decoder is no longer needed. The proposed RegProxy model discards the common Cartesian feature layout and operates purely at region level. Hence, it exhibits the most competitive performance-efficiency trade-off compared with the conventional dense prediction methods. For example, on ADE20K, the small-sized RegProxy-S/16 outperforms the best CNN model using 25% parameters and 4% computation, while the largest RegProxy-L/16 achieves 52.9mIoU which outperforms the state-of-the-art by 2.1% with fewer resources. Codes and models are available at https://github.com/YiF-Zhang/RegionProxy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑