AirSpatialBot: 一种空间感知的空中智能体用于细粒度车辆属性识别与检索
AirSpatialBot: A Spatially-Aware Aerial Agent for Fine-Grained Vehicle Attribute Recognization and Retrieval
AI总结：
AirSpatialBot通过空间感知的VLM和两阶段训练策略，实现细粒度车辆属性识别与检索，解决遥感图像中的空间理解难题。
摘要：
尽管在遥感视觉-语言模型（VLMs）方面取得了显著进展，现有模型往往在空间理解上存在困难，限制了其在现实中的应用效果。为了推动VLMs在遥感领域的边界，我们专门针对无人机拍摄的车辆图像，引入了一个空间感知的数据集AirSpatial，其中包含超过206,000条指令，并引入了两个新的任务：空间定位和空间问答。它也是第一个提供3DBB的遥感定位数据集。为了有效地利用现有图像理解能力扩展到空间领域，我们采用了一种两阶段训练策略，包括图像理解预训练和空间理解微调。利用这个训练好的空间感知VLM，我们开发了空中智能体AirSpatialBot，能够进行细粒度车辆属性识别和检索。通过动态整合任务规划、图像理解、空间理解和任务执行能力，AirSpatialBot能够适应多样的查询需求。实验结果验证了我们方法的有效性，揭示了现有VLMs的空间局限性，并提供了有价值的见解。模型、代码和数据集将在https://github.com/VisionXLab/AirSpatialBot上发布。
英文摘要：
Despite notable advancements in remote sensing vision-language models (VLMs), existing models often struggle with spatial understanding, limiting their effectiveness in real-world applications. To push the boundaries of VLMs in remote sensing, we specifically address vehicle imagery captured by drones and introduce a spatially-aware dataset AirSpatial, which comprises over 206K instructions and introduces two novel tasks: Spatial Grounding and Spatial Question Answering. It is also the first remote sensing grounding dataset to provide 3DBB. To effectively leverage existing image understanding of VLMs to spatial domains, we adopt a two-stage training strategy comprising Image Understanding Pre-training and Spatial Understanding Fine-tuning. Utilizing this trained spatially-aware VLM, we develop an aerial agent, AirSpatialBot, which is capable of fine-grained vehicle attribute recognition and retrieval. By dynamically integrating task planning, image understanding, spatial understanding, and task execution capabilities, AirSpatialBot adapts to diverse query requirements. Experimental results validate the effectiveness of our approach, revealing the spatial limitations of existing VLMs while providing valuable insights. The model, code, and datasets will be released at https://github.com/VisionXLab/AirSpatialBot
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
