arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.03465cs.CVeess.IV

DTU-Net:一种用于非线性高光谱解混的多尺度膨胀Transformer网络

DTU-Net: A Multi-Scale Dilated Transformer Network for Nonlinear Hyperspectral Unmixing

Tianjin University(天津大学);Heriot-Watt University(赫瑞瓦特大学);CNRS(法国国家科学研究中心);OCA(蔚蓝海岸天文台)

ChenTong Wang, Jincheng Gao, Fei Zhu, Abderrahim Halimi, Cédric Richard

AI总结:

提出DTU-Net,一种用于非线性高光谱解混的多尺度膨胀Transformer网络,通过双分支编码器融合多尺度空间与光谱特征,并结合PPNMM建模提升了解混精度与可解释性。

摘要:

Transformers在高光谱解混(HU)中取得了显著成功。然而,挑战依然存在。虽然多尺度和长程空间相关性在解混任务中至关重要,但当前基于Transformer的解混网络(构建于Vision Transformer (ViT)或Swin-Transformer之上)难以有效捕获它们。此外,当前基于Transformer的解混网络依赖线性混合模型,缺乏灵活性以适应非线性效应显著的场景。为解决这些局限性,我们提出了一种用于非线性HU的多尺度膨胀Transformer解混网络(DTU-Net)。编码器采用两个分支。第一个分支在Dilated Transformer中使用多尺度膨胀注意力(MSDA)执行多尺度空间特征提取,该机制在不同注意力头间改变膨胀率以捕获长程和多尺度空间相关性。第二个分支利用带通道注意力的3D-CNNs执行光谱特征提取。随后融合两个分支的输出以整合多尺度空间和光谱信息,并对其进行变换以估计丰度。解码器旨在适应线性和非线性混合场景。通过依据多项式后非线性混合模型(PPNMM)显式建模端元、丰度和非线性系数之间的关系,增强了其可解释性。在合成和真实数据集上的实验验证了所提出的DTU-Net相较于PPNMM衍生方法和几种先进解混网络的有效性。

英文摘要:

Transformers have shown significant success in hyperspectral unmixing (HU). However, challenges remain. While multi-scale and long-range spatial correlations are essential in unmixing tasks, current Transformer-based unmixing networks, built on Vision Transformer (ViT) or Swin-Transformer, struggle to capture them effectively. Additionally, current Transformer-based unmixing networks rely on the linear mixing model, which lacks the flexibility to accommodate scenarios where nonlinear effects are significant. To address these limitations, we propose a multi-scale Dilated Transformer-based unmixing network for nonlinear HU (DTU-Net). The encoder employs two branches. The first one performs multi-scale spatial feature extraction using Multi-Scale Dilated Attention (MSDA) in the Dilated Transformer, which varies dilation rates across attention heads to capture long-range and multi-scale spatial correlations. The second one performs spectral feature extraction utilizing 3D-CNNs with channel attention. The outputs from both branches are then fused to integrate multi-scale spatial and spectral information, which is subsequently transformed to estimate the abundances. The decoder is designed to accommodate both linear and nonlinear mixing scenarios. Its interpretability is enhanced by explicitly modeling the relationships between endmembers, abundances, and nonlinear coefficients in accordance with the polynomial post-nonlinear mixing model (PPNMM). Experiments on synthetic and real datasets validate the effectiveness of the proposed DTU-Net compared to PPNMM-derived methods and several advanced unmixing networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑