面向多光谱卫星图像的星上全景分割
Towards On-Board Panoptic Segmentation of Multispectral Satellite Images
AI总结：
针对多光谱卫星图像星上全景分割需求，提出基于跨模态注意力融合的多模态教师网络与在线知识蒸馏框架，将知识迁移至单模态学生网络适配星上环境，在PASTIS数据集上精度显著优于现有SOTA模型。
摘要：
随着低功耗嵌入式计算设备与遥感仪器的巨大进步，传统卫星图像处理流程（需先经过成本高昂的数据传输步骤再在地面处理数据）正被捕获数据的星上处理所取代。这种范式转变使得关键且时效性强的分析情报能够在卫星本身上及时获取。但目前，多光谱卫星图像的星上处理仅限于分类和分割任务。为将该处理推进至下一个合理阶段，本文提出一种轻量级流程，用于多光谱卫星图像的星上全景分割。全景分割可提供重要的经济与环境洞察，涵盖从农田产量估算到复杂军事应用情报等多个方面。然而，由于缺少时间序列观测且需基于单幅图像样本生成预测，星上情报提取面临若干挑战。为应对这一挑战，我们提出一种基于跨模态注意力融合策略的多模态教师网络，通过利用多模态数据提升分割精度。我们还提出一种在线知识蒸馏框架，将该多模态教师网络学到的知识迁移到仅接收单帧输入的单模态学生网络中，该学生网络更适配星上环境。我们在PASTIS多光谱全景分割数据集上，结合星上处理设置，将所提方法与现有最先进的全景分割模型进行基准测试。评估结果表明，与现有最先进模型相比，所提方法的精度指标有大幅提升。
英文摘要：
With tremendous advancements in low-power embedded computing devices and remote sensing instruments, the traditional satellite image processing pipeline which includes an expensive data transfer step prior to processing data on the ground is being replaced by on-board processing of captured data. This paradigm shift enables critical and time-sensitive analytic intelligence to be acquired in a timely manner on-board the satellite itself. However, at present, the on-board processing of multi-spectral satellite images is limited to classification and segmentation tasks. Extending this processing to its next logical level, in this paper we propose a lightweight pipeline for on-board panoptic segmentation of multi-spectral satellite images. Panoptic segmentation offers major economic and environmental insights, ranging from yield estimation from agricultural lands to intelligence for complex military applications. Nevertheless, the on-board intelligence extraction raises several challenges due to the loss of temporal observations and the need to generate predictions from a single image sample. To address this challenge, we propose a multimodal teacher network based on a cross-modality attention-based fusion strategy to improve the segmentation accuracy by exploiting data from multiple modes. We also propose an online knowledge distillation framework to transfer the knowledge learned by this multi-modal teacher network to a uni-modal student which receives only a single frame input, and is more appropriate for an on-board environment. We benchmark our approach against existing state-of-the-art panoptic segmentation models using the PASTIS multi-spectral panoptic segmentation dataset considering an on-board processing setting. Our evaluations demonstrate a substantial increase in accuracy metrics compared to the existing state-of-the-art models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
