利用大型语言模型实现卫星图像标题自动生成
Towards Automatic Satellite Images Captions Generation Using Large Language Models
AI总结：
针对遥感图像标题数据稀缺及现有模型适配性差的问题，提出ARSIC方法，利用大型语言模型自动生成标题，并基于GIT构建基准模型，实验验证了其有效性。
摘要：
自动图像标题生成是一种利用自然语言传达视觉信息的有前景的技术。它可以使卫星遥感中的各种任务受益，例如环境监测、资源管理、灾害管理等。然而，该领域的主要挑战之一是缺乏大规模图像-标题数据集，因为创建这些数据集需要大量的人类专业知识和努力。近期关于大型语言模型（LLMs）的研究已展示了它们在自然语言理解和生成任务中的出色表现。尽管如此，大多数此类模型无法处理图像（如GPT-3.5、Falcon、Claude等），而预训练于一般地面视角图像的传统标题生成模型（如BLIP、GIT、CM3、CM3Leon等）往往无法为航拍图像生成详细且准确的标题。为解决这一问题，我们提出了一种新颖的方法：自动遥感图像标题生成（ARSIC），通过引导大型语言模型描述其对象标注来自动收集遥感图像的标题。我们还提出了一个基准模型，该模型改编了预训练的生成式图像到文本模型（GIT），以为遥感图像生成高质量标题。我们的评估证明了我们方法在收集遥感图像标题方面的有效性。
英文摘要：
Automatic image captioning is a promising technique for conveying visual information using natural language. It can benefit various tasks in satellite remote sensing, such as environmental monitoring, resource management, disaster management, etc. However, one of the main challenges in this domain is the lack of large-scale image-caption datasets, as they require a lot of human expertise and effort to create. Recent research on large language models (LLMs) has demonstrated their impressive performance in natural language understanding and generation tasks. Nonetheless, most of them cannot handle images (GPT-3.5, Falcon, Claude, etc.), while conventional captioning models pre-trained on general ground-view images often fail to produce detailed and accurate captions for aerial images (BLIP, GIT, CM3, CM3Leon, etc.). To address this problem, we propose a novel approach: Automatic Remote Sensing Image Captioning (ARSIC) to automatically collect captions for remote sensing images by guiding LLMs to describe their object annotations. We also present a benchmark model that adapts the pre-trained generative image2text model (GIT) to generate high-quality captions for remote-sensing images. Our evaluation demonstrates the effectiveness of our approach for collecting captions for remote sensing images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
