AeroLite：标签引导的轻量化航空图像描述生成
AeroLite: Tag-Guided Lightweight Generation of Aerial Image Captions
AI总结：
针对航空图像描述的空间语义复杂和领域变异性问题，提出轻量化标签引导框架AeroLite，通过伪数据集生成、语义标签提取与桥接MLP融合，使小模型（1-3B参数）在遥感图像描述任务上超越大模型（13B参数），且计算成本更低。
摘要：
航空图像的准确自动描述对于环境监测、城市规划和灾害管理等应用至关重要。然而，由于复杂的空间语义和领域变异性，该任务仍具挑战性。为解决这些问题，我们引入AeroLite——一个轻量化、标签引导的描述框架，旨在为小规模语言模型（1-3B参数）配备针对遥感图像的鲁棒且可解释的描述能力。AeroLite利用GPT-4o，通过整合DLRSD、iSAID、LoveDA、WHU和RSSCN7等多个遥感基准，生成大规模、语义丰富的伪描述数据集。为明确捕捉方向和土地利用类型等关键语义元素，AeroLite采用自然语言处理技术提取相关语义标签；这些标签由专用多标签CLIP编码器学习，确保精确的语义预测。为有效融合视觉和语义信息，我们提出新颖的桥接多层感知器（MLP）架构，在保持最小计算开销的同时对齐语义标签与视觉嵌入。AeroLite的灵活设计还支持与各种预训练大型语言模型无缝集成。我们采用两阶段基于LoRA的训练方法：初始阶段利用伪描述数据集捕捉广泛的遥感语义，随后在UCM和Sydney Captions等较小的精选数据集上微调以优化领域特定对齐。实验评估表明，AeroLite在BLEU和METEOR等标准描述指标上超过大得多的模型（如13B参数），同时保持显著更低的计算成本。
英文摘要：
Accurate and automated captioning of aerial imagery is crucial for applications like environmental monitoring, urban planning, and disaster management. However, this task remains challenging due to complex spatial semantics and domain variability. To address these issues, we introduce \textbf{AeroLite}, a lightweight, tag-guided captioning framework designed to equip small-scale language models (1--3B parameters) with robust and interpretable captioning capabilities specifically for remote sensing images. \textbf{AeroLite} leverages GPT-4o to generate a large-scale, semantically rich pseudo-caption dataset by integrating multiple remote sensing benchmarks, including DLRSD, iSAID, LoveDA, WHU, and RSSCN7. To explicitly capture key semantic elements such as orientation and land-use types, AeroLite employs natural language processing techniques to extract relevant semantic tags. These tags are then learned by a dedicated multi-label CLIP encoder, ensuring precise semantic predictions. To effectively fuse visual and semantic information, we propose a novel bridging multilayer perceptron (MLP) architecture, aligning semantic tags with visual embeddings while maintaining minimal computational overhead. AeroLite's flexible design also enables seamless integration with various pretrained large language models. We adopt a two-stage LoRA-based training approach: the initial stage leverages our pseudo-caption dataset to capture broad remote sensing semantics, followed by fine-tuning on smaller, curated datasets like UCM and Sydney Captions to refine domain-specific alignment. Experimental evaluations demonstrate that AeroLite surpasses significantly larger models (e.g., 13B parameters) in standard captioning metrics, including BLEU and METEOR, while maintaining substantially lower computational costs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
