寻求共识:面向开放词汇遥感语义分割的几何-语义即插即用重校准
Seeking Consensus: Geometric-Semantic On-the-Fly Recalibration for Open-Vocabulary Remote Sensing Semantic Segmentation
AI总结:
本文提出SeeCo框架,通过几何和语义双共识学习提升遥感开放词汇语义分割性能,无需训练即可实现场景特异性重校准。
摘要:
开放词汇语义分割(OVSS)在遥感图像中是一项有前景的任务,利用文本描述识别未定义的土地覆盖类别。尽管取得了显著进展,现有方法通常采用静态推理范式,忽视了每个场景的分布差异,导致多样化的土地覆盖存在语义模糊和前景激活不完整的问题。受此启发,我们提出Seeking Consensus(SeeCo),一种即插即用框架,旨在提升遥感图像中无需训练的OVSS模型性能。SeeCo通过即插即用的在线共识注入器(OCI)实现双共识:通过多视角一致观测进行几何共识学习(GCL),并通过文本描述自适应校准进行语义共识学习(SCL),从而协助视觉和文本语义的协同重校准。两种共识通过OCI注入,有效缓解了欠激活和语义偏差问题。SeeCo无需特定训练过程,但能在推理过程中为每个独特场景重校准语义-几何对齐。在八个遥感OVSS基准上的广泛实验显示了一致的提升,证明了其有效性和通用性。
英文摘要:
Open-vocabulary semantic segmentation (OVSS) in remote sensing images is a promising task that employs textual descriptions for identifying undefined land cover categories. Despite notable advances, existing methods typically employ a static inference paradigm, overlooking the distinct distribution of each scene, resulting in semantic ambiguity in diverse land covers and incomplete foreground activation. Motivated by this, we propose Seeking Consensus, termed SeeCo, a plug-and-play framework to boost the performance of training-free OVSS models in remote sensing images, which recalibrates arbitrary OVSS models on-the-fly by seeking dual consensus: geometric consensus learning (GCL) through multi-view consistent observations and semantic consensus learning (SCL) via textual description adaptive calibration, which assists collaborative recalibration of visual and textual semantics. The two consensus are injected via an online consensus injector (OCI), effectively alleviating the under-activation and semantic bias. SeeCo requires no specific training process, yet recalibrates semantic-geometric alignment for each unique scene during inference. Extensive experiments on eight remote sensing OVSS benchmarks show consistent gains, proving its effectiveness and universality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。