arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.08655cs.CV
收藏

语义分割中隐式特征对齐函数的学习

Learning Implicit Feature Alignment Function for Semantic Segmentation

Peking University(北京大学);UC San Diego(加州大学圣地亚哥分校);Qualcomm AI Research(高通人工智能研究院)

Hanzhe Hu, Yinbo Chen, Jiarui Xu, Shubhankar Borse, Hong Cai, Fatih Porikli, Xiaolong Wang

摘要:

在语义分割中,将高层上下文信息与低层细节相结合是至关重要的。为此,大多数现有的分割模型对不同尺度的特征图应用双线性上采样和卷积,然后以相同的分辨率对齐它们。然而,双线性上采样模糊了在这些特征映射中学习到的精确信息,卷积会产生额外的计算成本。为了解决这些问题,我们提出了隐式特征对齐功能(IFA)。我们的方法的灵感来自于隐式神经表示的快速扩展的主题,其中基于坐标的神经网络用于指定信号的字段。在IFA中,特征向量被视为表示2D信息场。给定一个查询坐标,附近的特征向量及其相对坐标从多级特征图中提取,然后送入MLP以生成相应的输出。因此,IFA隐式地对齐不同级别的特征图,并且能够以任意分辨率生成分割图。我们证明了IFA在多个数据集上的有效性,包括Cityscapes,PASCAL Context和ADE20K。我们的方法可以结合各种架构的改进,它实现了最先进的计算精度的权衡常见的基准。代码将在https://github.com/hzhupku/IFA上提供。

英文摘要:

Integrating high-level context information with low-level details is of central importance in semantic segmentation. Towards this end, most existing segmentation models apply bilinear up-sampling and convolutions to feature maps of different scales, and then align them at the same resolution. However, bilinear up-sampling blurs the precise information learned in these feature maps and convolutions incur extra computation costs. To address these issues, we propose the Implicit Feature Alignment function (IFA). Our method is inspired by the rapidly expanding topic of implicit neural representations, where coordinate-based neural networks are used to designate fields of signals. In IFA, feature vectors are viewed as representing a 2D field of information. Given a query coordinate, nearby feature vectors with their relative coordinates are taken from the multi-level feature maps and then fed into an MLP to generate the corresponding output. As such, IFA implicitly aligns the feature maps at different levels and is capable of producing segmentation maps in arbitrary resolutions. We demonstrate the efficacy of IFA on multiple datasets, including Cityscapes, PASCAL Context, and ADE20K. Our method can be combined with improvement on various architectures, and it achieves state-of-the-art computation-accuracy trade-off on common benchmarks. Code will be made available at https://github.com/hzhupku/IFA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑