上下文确实重要：基于可变形注意力精炼匹配网络的端到端全景叙事定位
Context Does Matter: End-to-end Panoptic Narrative Grounding with Deformable Attention Refined Matching Network
AI总结：
针对全景叙事定位中忽略上下文导致短语与像素错配的问题，提出可变形注意力精炼匹配网络（DRMN），通过迭代式引入可变形注意力整合多尺度上下文信息，在PNG基准上实现3.5%的平均召回率提升。
摘要：
全景叙事定位（PNG）是一项新兴的视觉定位任务，旨在根据密集的叙事描述对图像中的视觉对象进行分割。当前最先进的方法首先通过聚合最相似的k个图像像素来精炼短语表示，然后将精炼后的文本表示与图像特征图的像素进行匹配以生成分割结果。然而，简单地聚合采样图像特征忽略了上下文信息，这可能导致短语到像素的错配。在本文中，我们提出了一种名为可变形注意力精炼匹配网络（DRMN）的新型学习框架，其核心思想是在特征学习的迭代过程中引入可变形注意力，以整合不同尺度像素的关键上下文信息。DRMN在更新最相似的top-k个像素的特征表示后，使用可变形注意力网络对像素进行迭代式重新编码。如此一来，DRMN能够生成准确且具判别性的像素表示，净化最相似的top-k个像素，从而显著缓解短语到像素的错配问题。实验结果表明，我们的创新设计显著提升了文本短语与图像像素之间的匹配效果。具体而言，DRMN在PNG基准上取得了新的最优性能，平均召回率提升了3.5%。代码可在以下地址获取：https://github.com/JaMesLiMers/DRMN。
英文摘要：
Panoramic Narrative Grounding (PNG) is an emerging visual grounding task that aims to segment visual objects in images based on dense narrative captions. The current state-of-the-art methods first refine the representation of phrase by aggregating the most similar $k$ image pixels, and then match the refined text representations with the pixels of the image feature map to generate segmentation results. However, simply aggregating sampled image features ignores the contextual information, which can lead to phrase-to-pixel mis-match. In this paper, we propose a novel learning framework called Deformable Attention Refined Matching Network (DRMN), whose main idea is to bring deformable attention in the iterative process of feature learning to incorporate essential context information of different scales of pixels. DRMN iteratively re-encodes pixels with the deformable attention network after updating the feature representation of the top-$k$ most similar pixels. As such, DRMN can lead to accurate yet discriminative pixel representations, purify the top-$k$ most similar pixels, and consequently alleviate the phrase-to-pixel mis-match substantially.Experimental results show that our novel design significantly improves the matching results between text phrases and image pixels. Concretely, DRMN achieves new state-of-the-art performance on the PNG benchmark with an average recall improvement 3.5%. The codes are available in: https://github.com/JaMesLiMers/DRMN.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
