解释多模态数据融合：面向荒野地图制作的遮挡分析
Explaining Multimodal Data Fusion: Occlusion Analysis for Wilderness Mapping
AI总结：
本研究提出一种深度学习框架，利用遮挡敏感性方法分析早期融合场景下各模态对模型决策的影响，发现荒野地图制作任务显著受益于土地覆盖和夜间灯光等辅助数据。
摘要：
在共同潜在空间中联合利用多模态输入数据的互补特征早已被发现是有益的。然而，每种模态对模型决策的影响仍是一个谜。本研究提出了一种深度学习框架，以端到端方式对多模态地球观测数据进行模态级解释。在利用一种可解释的机器学习方法（即遮挡敏感性，Occlusion Sensitivity）的同时，该框架研究了早期融合场景下各模态的影响力，即在学习过程之前融合各模态。我们表明，荒野地图制作任务在很大程度上受益于辅助数据，如土地覆盖和夜间灯光数据。
英文摘要：
Jointly harnessing complementary features of multi-modal input data in a common latent space has been found to be beneficial long ago. However, the influence of each modality on the models decision remains a puzzle. This study proposes a deep learning framework for the modality-level interpretation of multimodal earth observation data in an end-to-end fashion. While leveraging an explainable machine learning method, namely Occlusion Sensitivity, the proposed framework investigates the influence of modalities under an early-fusion scenario in which the modalities are fused before the learning process. We show that the task of wilderness mapping largely benefits from auxiliary data such as land cover and night time light data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
