SegVG：将目标边界框迁移为分割以用于视觉定位
SegVG: Transferring Object Bounding Box to Segmentation for Visual Grounding
AI总结：
本文提出SegVG方法，将框级标注迁移为分割信号，通过多层多任务编码器-解码器和三重对齐模块，为视觉定位提供像素级监督并缓解特征域差异，在多个数据集上取得SOTA性能。
摘要：
与目标检测不同，视觉定位处理的是为每个文本-图像对检测一个边界框。每个文本-图像数据仅有一个框，这提供了稀疏的监督信号。尽管先前的工作取得了令人瞩目的成果，但它们对标注的被动利用，即仅将框标注用作回归的真值，导致了次优的性能。在本文中，我们提出了SegVG，这是一种新颖的方法，将框级标注迁移为分割信号，从而为视觉定位提供额外的像素级监督。具体而言，我们提出了多层多任务编码器-解码器作为目标定位阶段，在该阶段中，我们学习一个回归查询和多个分割查询，分别在每个解码层中通过回归和框的分割来定位目标。这种方法使我们能够迭代地利用标注作为框级回归和像素级分割的信号。此外，由于骨干网络通常由从单模态任务中学习到的预训练参数初始化，并且回归和分割的查询都是静态可学习嵌入，这三种类型的特征之间仍然存在域差异，这会损害后续的目标定位。为了缓解这种差异，我们引入了三重对齐模块，其中查询、文本和视觉token通过三重注意力机制进行三角更新，以共享相同的空间。在五个广泛使用的数据集上进行的大量实验验证了我们最先进的（SOTA）性能。
英文摘要：
Different from Object Detection, Visual Grounding deals with detecting a bounding box for each text-image pair. This one box for each text-image data provides sparse supervision signals. Although previous works achieve impressive results, their passive utilization of annotation, i.e. the sole use of the box annotation as regression ground truth, results in a suboptimal performance. In this paper, we present SegVG, a novel method transfers the box-level annotation as Segmentation signals to provide an additional pixel-level supervision for Visual Grounding. Specifically, we propose the Multi-layer Multi-task Encoder-Decoder as the target grounding stage, where we learn a regression query and multiple segmentation queries to ground the target by regression and segmentation of the box in each decoding layer, respectively. This approach allows us to iteratively exploit the annotation as signals for both box-level regression and pixel-level segmentation. Moreover, as the backbones are typically initialized by pretrained parameters learned from unimodal tasks and the queries for both regression and segmentation are static learnable embeddings, a domain discrepancy remains among these three types of features, which impairs subsequent target grounding. To mitigate this discrepancy, we introduce the Triple Alignment module, where the query, text, and vision tokens are triangularly updated to share the same space by triple attention mechanism. Extensive experiments on five widely used datasets validate our state-of-the-art (SOTA) performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
