arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.00629cs.CVeess.IV

面向鲁棒光谱成像的带掩码训练的跨扫描Mamba

Cross-Scan Mamba with Masked Training for Robust Spectral Imaging

University of Helsinki(赫尔辛基大学);IMEC(微电子中心);Universiteit Gent(根特大学);Northwestern Polytechnical University(西北工业大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))

Wenzhe Tian, Haijin Zeng, Yin-Ping Zhao, Yongyong Chen, Zhen Wang, Xuelong Li

AI总结:

针对高光谱图像重建中现有方法长依赖建模不足、泛化性差的问题,提出带掩码训练的跨扫描Mamba模型,通过空-谱SSM实现全局-局部平衡编码,取得SOTA性能并提升真实场景重建视觉质量。

摘要:

快照压缩成像(Snapshot Compressive Imaging, SCI)可实现快速光谱成像,但需要有效的解码算法从压缩测量值中重建高光谱图像(Hyperspectral Image, HSI)。当前基于CNN的方法在建模长距离依赖关系方面存在局限,而基于Transformer的模型则面临计算复杂度高的问题。尽管近期的Mamba模型在RGB任务的计算效率或精度上优于CNN和Transformer,但它们未经过专门优化,无法充分利用HSI固有的局部空间和光谱相关性。为解决这一问题,我们提出跨扫描Mamba(命名为CS-Mamba),该模型采用空-谱状态空间模型(Spatial-Spectral SSM)实现全局-局部平衡的上下文编码,并促进通道间交互。此外,尽管当前重建算法在模拟场景中的表现越来越好,但由于泛化能力有限,它们在真实数据上的表现欠佳。在训练过程中,模型可能无法捕捉图像的固有特征,反而会学习缓解特定噪声和损失的参数,这可能导致模型在面对真实场景时重建质量下降。为克服这一挑战,我们提出一种掩码训练方法以增强模型的泛化能力。实验结果表明,我们的CS-Mamba取得了当前最优性能,且掩码训练方法能够更好地重建平滑特征,提升视觉质量。

英文摘要:

Snapshot Compressive Imaging (SCI) enables fast spectral imaging but requires effective decoding algorithms for hyperspectral image (HSI) reconstruction from compressed measurements. Current CNN-based methods are limited in modeling long-range dependencies, while Transformer-based models face high computational complexity. Although recent Mamba models outperform CNNs and Transformers in RGB tasks concerning computational efficiency or accuracy, they are not specifically optimized to fully leverage the local spatial and spectral correlations inherent in HSIs. To address this, we propose the Cross-Scanning Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local balanced context encoding and cross-channel interaction promotion. Besides, while current reconstruction algorithms perform increasingly well in simulation scenarios, they exhibit suboptimal performance on real data due to limited generalization capability. During the training process, the model may not capture the inherent features of the images but rather learn the parameters to mitigate specific noise and loss, which may lead to a decline in reconstruction quality when faced with real scenes. To overcome this challenge, we propose a masked training method to enhance the generalization ability of models. Experiment results show that our CS-Mamba achieves state-of-the-art performance and the masked training method can better reconstruct smooth features to improve the visual quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑