arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.10596cs.CVcs.AI
收藏

超越像素:一种无训练的文本到文本框架用于遥感图像检索

Beyond Pixels: A Training-Free, Text-to-Text Framework for Remote Sensing Image Retrieval

Information Technology University of Technology Sydney Sydney, Australia(信息科技技术大学悉尼分校悉尼澳大利亚);Robotics Laboratory (SensR Lab) Centre for Advanced Manufacturing Technology Western Sydney University Sydney, Australia(机器人实验室(SensR实验室)先进制造技术中心西悉尼大学悉尼澳大利亚);The Data Science Institute Faculty of Engineering(数据科学学院工程学院)

J. Xiao, Y. Guo, X. Zi, K. Thiyagarajan, C. Moreira, M. Prasad

AI总结:

本文提出无训练的文本到文本框架TRSLLaVA,通过结构化文本实现高效遥感图像检索,实验表明其性能优于现有监督模型。

摘要:

遥感图像的语义检索是一项关键任务,根本上受到'语义鸿沟'的挑战,即模型的低级视觉特征与人类高级概念之间的差异。尽管大型视觉语言模型(VLMs)提供了一条有希望的途径来弥合这一差距,但现有方法通常依赖于成本高昂、领域特定的训练,并缺乏评估VLM生成文本在零样本检索上下文中的实用性的基准。为了解决这一研究空白,我们引入了遥感丰富文本(RSRT)数据集,这是一个新基准,每个图像具有多个结构化描述。基于此数据集,我们提出了一种完全无训练的、仅文本的检索参考称为TRSLLaVA。我们的方法将跨模态检索重新表述为文本到文本(T2T)匹配问题,利用丰富的文本描述作为查询,与包含VLM生成描述的数据库在统一的文本嵌入空间中进行匹配。这种方法完全绕过了模型训练或微调。在RSITMD和RSICD基准上的实验表明,我们的无训练方法与最先进的监督模型高度竞争。例如,在RSITMD上,我们的方法实现了42.62%的平均召回率,几乎翻倍了标准零样本CLIP基线的23.86%,并超过了几个顶级监督模型。这验证了高质量的语义表示通过结构化文本提供了一种强大而经济有效的遥感图像检索范式。

英文摘要:

Semantic retrieval of remote sensing (RS) images is a critical task fundamentally challenged by the \textquote{semantic gap}, the discrepancy between a model's low-level visual features and high-level human concepts. While large Vision-Language Models (VLMs) offer a promising path to bridge this gap, existing methods often rely on costly, domain-specific training, and there is a lack of benchmarks to evaluate the practical utility of VLM-generated text in a zero-shot retrieval context. To address this research gap, we introduce the Remote Sensing Rich Text (RSRT) dataset, a new benchmark featuring multiple structured captions per image. Based on this dataset, we propose a fully training-free, text-only retrieval reference called TRSLLaVA. Our methodology reformulates cross-modal retrieval as a text-to-text (T2T) matching problem, leveraging rich text descriptions as queries against a database of VLM-generated captions within a unified textual embedding space. This approach completely bypasses model training or fine-tuning. Experiments on the RSITMD and RSICD benchmarks show our training-free method is highly competitive with state-of-the-art supervised models. For instance, on RSITMD, our method achieves a mean Recall of 42.62\%, nearly doubling the 23.86\% of the standard zero-shot CLIP baseline and surpassing several top supervised models. This validates that high-quality semantic representation through structured text provides a powerful and cost-effective paradigm for remote sensing image retrieval.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑