arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.20973cs.CV
收藏

PiCo:面向地理基础模型的物理信息概念探测

PiCo: Physics-Informed Concept Probing for Geo-Foundation Models

University of Cambridge(剑桥大学);Beijing Normal University(北京师范大学)

Yihang She, Andrew Blake, Jianbo Qi, Srinivasan Keshav

AI总结:

PiCo是解释地理基础模型(GFM)预测的事后框架,通过合成多光谱卫星观测识别概念向量,在3种GFM的3类任务中验证了其用于物理敏感性分析和模型评估的有效性。

摘要:

地理基础模型(GFMs)通常通过下游任务的预测性能进行评估,这几乎无法揭示它们在预测中对哪些物理特征敏感或使用了哪些物理特征。这限制了开发者测试模型行为假设的能力,也限制了用户为新任务选择模型的能力。我们提出了物理信息概念探测(PiCo),这是一种用于解释GFM预测的事后框架。PiCo通过将3D辐射传输模型与游戏引擎耦合生成合成多光谱卫星观测,在GFM的嵌入空间中识别概念向量。这些向量揭示了物理敏感性,为基于领域的下游性能假设提供依据。将嵌入投影到这些概念向量上会产生低维表示,用于量化概念在预测中的重要性。我们使用涵盖森林叶片光学和冠层结构的7个物理概念,对具有不同输入模态的3种GFM评估PiCo。在地面覆盖估计中,RGB限制会降低干物质敏感性,并不成比例地降低干燥植被和裸地的预测性能,这支持了从敏感性分析得出的假设。对于分层森林类型和树种分类,概念表示保留了完整嵌入准确率的70.0%-93.8%。模型捕捉到类别之间相似的物理对比,例如阔叶林与 cleared 地面的结构差异,以及橡树与山毛榉之间的叶片含水量差异,但在这些对比的强度和驱动分类的概念上存在差异。因此,PiCo为假设测试和模型评估提供了物理基础,在正向模型支持受控模拟且可获取预训练嵌入进行探测的场景中,其潜在应用超出了地球观测领域。

英文摘要:

Geo-foundation models (GFMs) are evaluated by downstream predictive performance. This reveals little about which physical properties their embeddings are sensitive to or which are used in predictions. This limits developers' ability to test hypotheses about model behaviour and users' ability to select models for new tasks. We propose Physics-Informed Concept Probing (PiCo), a post-hoc framework for explaining GFM predictions. PiCo identifies concept vectors in GFM embedding space using synthetic multispectral satellite observations from a 3D radiative transfer model coupled with a game engine. These vectors characterise GFM sensitivities to physical properties, informing domain-based hypotheses about downstream performance. Projecting embeddings onto concept vectors produces low-dimensional representations for quantifying concept importance in predictions. We evaluate PiCo on three GFMs with different input modalities using seven physical concepts covering forest leaf optics and canopy structure. In ground-cover estimation, RGB-only inputs reduce dry-matter sensitivity and disproportionately degrade predictions for dry vegetation and bare ground, as anticipated by PiCo's sensitivity analysis. For hierarchical forest type and tree species classification, concept representations retain 70.0--93.8% of full-embedding accuracy. Different GFMs capture physical differences between classes, such as canopy structure between broadleaf forest and cleared ground and leaf-water-content between oak and beech. Yet they differ in the strength of these contrasts and the concepts they use for classification. PiCo provides an interpretable basis for hypothesis testing and model assessment, with potential applicability beyond Earth observation wherever physical models enable controlled simulation and pretrained embeddings are accessible for probing. Code is available at https://github.com/yihshe/PiCo.git.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑