arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2509.18693cs.CV

MVT:基于分类对齐的土地覆盖标签的掩码导向视觉-语言模型

MVT: Mask-Grounded Vision-Language Models for Taxonomy-Aligned Land-Cover Tagging

HKUST(香港科技大学);JHU(约翰·霍普金斯大学);CUHKSZ(香港中文大学);NUS(新加坡国立大学);MUST(穆斯林大学)

Siyi Chen, Kai Wang, Weicong Pang, Ruiming Yang, Ziru Chen, Renjun Gao, Alexis Kai Hon Lau, Dasa Gu, Chenchen Zhang, Cheng Li

AI总结:

MVT通过三阶段框架结合类无关掩码证据与基于分类的场景解释,提升遥感土地覆盖标签的准确性和信息性。

摘要:

遥感中的土地覆盖理解日益要求类无关系统,这些系统能够在不同数据集间泛化,同时保持空间精度和可解释性。我们研究了在领域转移下以几何优先的发现和解释设置,其中候选区域以类无关的方式界定,监督避免使用词汇类名,通过匿名标识符实现。与开放集识别和开放世界学习不同,我们专注于将类无关的掩码证据与基于分类的场景解释相结合,而不是未知拒绝或持续类扩展。我们提出了MVT,一个三阶段框架,(i) 使用域适应的SAM2提取边界忠实的区域掩码,(ii) 通过双步LoRA微调多模态大语言模型进行掩码导向的语义标注和场景描述生成,(iii) 使用由分层专家评分校准的LLM-as-judge评分评估输出。在跨数据集分割转移(在OpenEarthMap上训练,在LoveDA上评估)中,域适应的SAM2提高了掩码质量;同时,双步MLLM微调产生了更准确的分类对齐标签和更信息丰富的掩码导向场景描述。

英文摘要:

Land-cover understanding in remote sensing increasingly demands class-agnostic systems that generalize across datasets while remaining spatially precise and interpretable. We study a geometry-first discovery-and-interpretation setting under domain shift, where candidate regions are delineated class-agnostically and supervision avoids lexical class names via anonymized identifiers. Complementary to open-set recognition and open-world learning, we focus on coupling class-agnostic mask evidence with taxonomy-grounded scene interpretation, rather than unknown rejection or continual class expansion. We propose MVT, a three-stage framework that (i) extracts boundary-faithful region masks using SAM2 with domain adaptation, (ii) performs mask-grounded semantic tagging and scene description generation via dual-step LoRA fine-tuning of multimodal LLMs, and (iii) evaluates outputs with LLM-as-judge scoring calibrated by stratified expert ratings. On cross-dataset segmentation transfer (train on OpenEarthMap, evaluate on LoveDA), domain-adapted SAM2 improves mask quality; meanwhile, dual-step MLLM fine-tuning yields more accurate taxonomy-aligned tags and more informative mask-grounded scene descriptions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑