用于环境高光谱解混的智能体多模态模型

中文摘要 AI 辅助

高光谱解混是遥感领域的关键任务，旨在将高光谱图像中的混合像元分解为其组成物质的光谱特征（即端元）及其 fractional 丰度。传统模块化方法通过依次进行模型阶数估计、端元提取和丰度估计阶段来估计场景组成，其产生的误差可能导致冗余或模糊的候选成分，最终影响恢复的分解结果。我们提出一种与算法无关的、基于大视觉语言模型（LVLM）驱动的智能体框架，该框架用于优化此类模块化 pipeline 的输出，而非替换其底层数值算法。从初始分解结果出发，智能体通过专用工具迭代收集互补的光谱和空间证据，这些工具包括光谱库检索和丰度图可视化，并通过合并和丢弃操作修改活跃端元集，随后重新估计丰度。我们将相同的优化流程应用于多个结合了不同模型阶数、提取和丰度估计方法的模块化 pipeline，并在 HYDICE Urban、Jasper Ridge 和 Stonewall Playa 数据集上进行评估。实验表明，所提出的智能体在不同模块化 pipeline 中均能一致地改善端元基数，并普遍提升恢复的光谱特征和丰度图，同时与集成端到端解混方法（包括 CNN-AE、uDAS 和 R-CoNMF）具有竞争力。这些结果凸显了使用工具的 LVLM 智能体在结合光谱和空间证据以实现与算法无关的、基于物理的高光谱解混分解优化方面的潜力。代码可在该 https URL 公开获取。

英文摘要

Hyperspectral unmixing is a key task in remote sensing that aims to decompose mixed pixels in hyperspectral images into their constituent material signatures, or endmembers, and their fractional abundances. Conventional modular approaches estimate the scene composition through successive model-order estimation, endmember extraction, and abundance estimation stages, whose errors can lead to redundant or ambiguous candidate components and ultimately affect the recovered decomposition. We introduce an algorithm-agnostic, large vision-language model (LVLM)-driven agentic framework that refines the outputs of such pipelines rather than replacing their underlying numerical algorithms. Starting from an initial decomposition, the agent iteratively gathers complementary spectral and spatial evidence through dedicated tools, including spectral-library retrieval and abundance-map visualization, and modifies the active endmember set through merge and discard operations followed by abundance re-estimation. We apply the same refinement procedure to several modular pipelines combining different model-order, extraction, and abundance-estimation methods, and evaluate it on HYDICE Urban, Jasper Ridge, and Stonewall Playa. Experiments show that the proposed agent consistently improves endmember cardinality and generally improves the recovered spectral signatures and abundance maps across heterogeneous modular pipelines, while remaining competitive with integrated end-to-end unmixing methods, including CNN-AE, uDAS, and R-CoNMF. These results highlight the potential of tool-using LVLM agents to combine spectral and spatial evidence for algorithm-agnostic refinement of physically grounded hyperspectral unmixing decompositions. Code is publicly available at https://anonymous.4open.science/r/agentic-hu.

发表机构

• ISTI-CNR
• IITiS-PAS

机构由 AI 辅助整理，请以论文原文为准。