Earth-Adapter: 通过频率适应混合填补地理域的空白
Earth-Adapter: Bridge the Geospatial Domain Gaps with Mixture of Frequency Adaptation
AI总结：
Earth-Adapter通过频率适应混合方法有效解决遥感图像中的伪影问题，提升基础模型在遥感任务中的性能。
摘要：
参数高效微调（PEFT）是一种技术，使我们能够将强大的基础模型（FMs）适应到多样的下游任务，同时保留并释放其固有能力。然而，我们发现现有的PEFT方法，通常针对自然图像设计，难以应用于遥感（RS）场景。这主要是由于它们无法处理伪影影响，而这个问题在RS图像特征中尤为严重。为解决这一挑战，我们引入了Earth-Adapter，这是首个专门针对RS伪影的PEFT方法。Earth-Adapter引入了一种新颖的频率适应混合过程，将混合适配器（MoA）与离散傅里叶变换（DFT）相结合。通过利用DFT，Earth-Adapter可以将特征分解为不同的频率成分，精确分离伪影与原始特征。MoA随后动态地为每个适配器专家分配权重，允许在不同频率域内组合特征。这些简单而有效的做法使Earth-Adapter能够比先前的PEFT方法更高效地克服伪影造成的干扰，显著提升了FMs在RS场景中的性能。在域适应（DA）和域泛化（DG）语义分割基准上的实验展示了Earth-Adapter的有效性。与基线Rein相比，Earth-Adapter在DA基准上提高了9.0%的mIoU，在DG基准上提高了3.1%的mIoU。我们的代码将在https://github.com/VisionXLab/Earth-Adapter上发布。
英文摘要：
Parameter-Efficient Fine-Tuning (PEFT) is a technique that allows us to adapt powerful Foundation Models (FMs) to diverse downstream tasks while preserving and unleashing their inherent capabilities. However, we have observed that existing PEFT methods, which are often designed with natural imagery in mind, struggle when applied to Remote Sensing (RS) scenarios. This is primarily due to their inability to handle artifact influences, a problem particularly severe in RS image features. To tackle this challenge, we introduce Earth-Adapter, the first PEFT method specifically designed for RS artifacts conquering. Earth-Adapter introduces a novel Mixture of Frequency Adaptation process that combines a Mixture of Adapter (MoA) with Discrete Fourier Transformation (DFT). By utilizing DFT, Earth-Adapter can decompose features into different frequency components, precisely separating artifacts from original features. The MoA then dynamically assigns weights to each adapter expert, allowing for the combination of features across various frequency domains. These simple-yet-effective approaches enable Earth-Adapter to more efficiently overcome the disturbances caused by artifacts than previous PEFT methods, significantly enhancing the FMs' performance on RS scenarios. Experiments on Domain Adaptation (DA), and Domain Generalization (DG) semantic segmentation benchmarks showcase the Earth-Adapter's effectiveness. Compared with baseline Rein, Earth-Adapter significantly improves 9.0% mIoU in DA and 3.1% mIoU in DG benchmarks. Our code will be released at https://github.com/VisionXLab/Earth-Adapter.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
