发表机构
Fujian Institute of Research on the Structure of Matter, Chinese Academy of Sciences; Institute of Urban Environment, Chinese Academy of Sciences; University of Chinese Academy of Sciences; School of Resource and Environmental Sciences, Wuhan University（中国科学院福建物质结构研究所; 中国科学院城市环境研究所; 中国科学院大学; 武汉大学资源与环境科学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
COSTA提出以聚类为中心的范式，无需额外训练即可在推理阶段将预训练航拍点云分割模型适配到偏移目标域，在三个基准上实现按需分割，mIoU达70.09%。
AI 中文摘要
航拍点云的语义分割在不同域偏移下陷入泛化危机。尽管测试时自适应为在推理阶段将预训练模型适配到无标注目标域数据提供了一种隐私保护且计算高效的方式，但现有方法受限于闭集标签假设和不可扩展的逐点分割流程，仍难以应对语义偏移。本文提出的问题是：能否仅在推理阶段将任意给定的预训练航拍点云分割模型适配到偏移的目标域，无需额外训练，同时按需分割源标签空间之外的目标特定类别？本文引入COSTA，通过从闭集逐点自适应转向以聚类为中心的开放集语义传播打破了这一限制。核心发现是，一旦在测试时有效适配，航拍点云的丰富特征分布可被提炼为一组紧凑且分离良好的语义质心，这些质心可跨标签空间迁移。COSTA利用这一点将开放集语义分割重新表述为聚类级传播过程：首先通过成熟的测试时自适应弥合域差距，然后基于适配特征空间中的相似性分布，将每一批次的目标域点分组为少量语义聚类，最后通过聚类级投票将从开放词汇视觉语言模型获得的高置信度伪标签传播到所有点。这种以聚类为中心的范式可在存在显著域差距和混合语义偏移的情况下实现航拍点云的测试时自适应。以DALES作为源域，COSTA在三个具有不同域和异构类别空间的航拍点云基准上实现了按需分割，在该新设置下达到了70.09%的mIoU。
英文摘要
Semantic segmentation of aerial point cloud is trapped in a generalization crisis under distinct domain shifts. While test-time adaptation offers a privacy-preserving and computationally efficient way to adapt pre-trained models to unlabeled target-domain data during inference, existing methods, bound to closed-set label assumptions and non-scalable point-wise segmentation pipelines, still struggle with semantic shifts. We ask: can we adapt any given pre-trained aerial point cloud segmentation model to a shifted target domain at the inference phase alone, without additional training, while segmenting target-specific categories beyond the source label space on demand? This paper introduces COSTA, which breaks this limitation by shifting from closed-set point-wise adaptation to cluster-centric open-set semantic propagation. Our core discovery is that, once effectively adapted at test time, the rich feature distribution of aerial point clouds can be distilled into a compact set of well-separated semantic centroids that are transferable across label spaces. COSTA leverages this to reformulate open-set semantic segmentation as a cluster-level propagating process: it first bridges the domain gap through proven test-time adaptation, then groups each batch of target-domain points into a small set of semantic clusters based on the similarity distribution in the adapted feature space, and finally propagates high-confidence pseudo labels obtained from an open-vocabulary vision-language model to all points through cluster-level voting. This cluster-centric paradigm enables test-time adaptation of aerial point clouds under significant domain gaps with mixed semantic shifts. With DALES as the source domain, COSTA enables on-demand segmentation across three aerial point cloud benchmarks with distinct domains and heterogeneous category spaces, achieving up to 70.09% mIoU under this new setting.
