多语言免训练 遥感图像描述生成
Multilingual Training-Free Remote Sensing Image Captioning
AI总结：
本文提出一种无需训练的多语言遥感图像描述生成方法，通过检索增强提示和图基重新排序策略，实现跨语言的高效描述生成。
摘要：
遥感图像描述生成通过编码器-解码器模型迅速发展，尽管依赖于大规模标注数据集和专注于英语限制了全球适用性。为了解决这些限制，我们提出了首个无需训练的多语言方法，基于检索增强提示。对于给定的航空图像，我们使用经过领域适应的SigLIP2编码器从数据存储中检索相关描述和少量示例，然后将这些提供给语言模型。我们探索了两种变体：一种是图像盲设置，其中多语言大型语言模型（LLM）仅从文本提示生成描述；另一种是图像感知设置，其中视觉-语言模型（VLM）共同处理提示和输入图像。为了提高检索内容的连贯性，我们引入了一种基于图的重新排序策略，使用PageRank在图像和描述的图上进行重新排序。在四个基准数据集上进行的实验表明，我们的方法与完全监督的英语系统具有竞争力，并且能够推广到其他语言。结果还突显了使用PageRank重新排序的重要性，可使性能指标提高高达35%。此外，观察到虽然VLMs倾向于生成视觉基础但词汇多样的描述，而LLMs可以实现更强的BLEU和CIDEr分数。最后，直接生成目标语言的描述始终优于其他翻译策略。总体而言，我们的工作提供了首次系统评估多语言、无训练的遥感图像描述生成，推动了更包容和可扩展的多模态地球观测系统。
英文摘要：
Remote sensing image captioning has advanced rapidly through encoder--decoder models, although the reliance on large annotated datasets and the focus on English restricts global applicability. To address these limitations, we propose the first training-free multilingual approach, based on retrieval-augmented prompting. For a given aerial image, we employ a domain-adapted SigLIP2 encoder to retrieve related captions and few-shot examples from a datastore, which are then provided to a language model. We explore two variants: an image-blind setup, where a multilingual Large Language Model (LLM) generates the caption from textual prompts alone, and an image-aware setup, where a Vision--Language Model (VLM) jointly processes the prompt and the input image. To improve the coherence of the retrieved content, we introduce a graph-based re-ranking strategy using PageRank on a graph of images and captions. Experiments on four benchmark datasets across ten languages demonstrate that our approach is competitive with fully supervised English-only systems and generalizes to other languages. Results also highlight the importance of re-ranking with PageRank, yielding up to 35% improvements in performance metrics. Additionally, it was observed that while VLMs tend to generate visually grounded but lexically diverse captions, LLMs can achieve stronger BLEU and CIDEr scores. Lastly, directly generating captions in the target language consistently outperforms other translation-based strategies. Overall, our work delivers one of the first systematic evaluations of multilingual, training-free captioning for remote sensing imagery, advancing toward more inclusive and scalable multimodal Earth observation systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
