AeroReformer: 面向无人机指代图像分割的空中指代Transformer
AeroReformer: Aerial Referring Transformer for UAV-based Referring Image Segmentation
AI总结：
针对无人机影像指代图像分割研究不足、缺乏专用数据集的问题，提出带自动标注流水线和AeroReformer框架的方案，在新数据集上性能优于现有方法并建立新基准。
摘要：
指代分割是一项新颖且具有挑战性的任务，它结合计算机视觉与自然语言处理，根据文本描述定位并分割目标。尽管指代图像分割（RIS）在自然图像领域已得到广泛研究，但针对航空影像尤其是无人机（UAV）影像的相关研究却很少。无人机影像存在空间尺度复杂、遮挡问题以及目标朝向多变等独特挑战，导致现有RIS方法效果不佳。一个关键限制是缺乏无人机专用数据集，因为手动标注像素级掩码并生成文本描述既费力又耗时。为填补这一空白，我们设计了一套自动标注流水线，利用已有的无人机分割数据集和多模态大语言模型（MLLM）生成文本描述。此外，我们提出了空中指代Transformer（AeroReformer），这是一种用于无人机指代图像分割（UAV-RIS）的新型框架，其核心特性包括用于实现有效跨模态理解的视觉-语言交叉注意力模块（VLCAM），以及用于提升航空场景分割精度的旋转感知多尺度融合（RAMSF）解码器。在两个新构建的数据集上开展的大量实验表明，AeroReformer的性能优于现有方法，为UAV-RIS建立了新的基准。数据集和代码将公开发布于：https://github.com/lironui/AeroReformer。
英文摘要：
As a novel and challenging task, referring segmentation combines computer vision and natural language processing to localize and segment objects based on textual descriptions. While referring image segmentation (RIS) has been extensively studied in natural images, little attention has been given to aerial imagery, particularly from unmanned aerial vehicles (UAVs). The unique challenges of UAV imagery, including complex spatial scales, occlusions, and varying object orientations, render existing RIS approaches ineffective. A key limitation has been the lack of UAV-specific datasets, as manually annotating pixel-level masks and generating textual descriptions is labour-intensive and time-consuming. To address this gap, we design an automatic labelling pipeline that leverages pre-existing UAV segmentation datasets and Multimodal Large Language Models (MLLM) for generating textual descriptions. Furthermore, we propose Aerial Referring Transformer (AeroReformer), a novel framework for UAV referring image segmentation (UAV-RIS), featuring a Vision-Language Cross-Attention Module (VLCAM) for effective cross-modal understanding and a Rotation-Aware Multi-Scale Fusion (RAMSF) decoder to enhance segmentation accuracy in aerial scenes. Extensive experiments on two newly developed datasets demonstrate the superiority of AeroReformer over existing methods, establishing a new benchmark for UAV-RIS. The datasets and code will be publicly available at: https://github.com/lironui/AeroReformer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
