发表机构
Technical University of Denmark; University of Cambridge; University of Exeter（丹麦技术大学; 剑桥大学; 埃克塞特大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种通过物理表征空间中的预测结构评估机器学习模型动力学保真度的框架，并在大气预报中验证，发现传统误差未反映的模型偏离。
AI 中文摘要
机器学习模型用于物理系统时，目前主要通过预测状态与参考状态之间的误差以及日益增加的物理一致性测试来评估。这些指标评估预测是否准确并满足选定的物理要求，但对其学习到的轨迹是否再现了底层动力学提供的洞察有限。领域专家通过暴露相关过程、相互作用和响应的物理表征来检查此类关系，但这些分析通常与典型的机器学习评估相分离。我们引入了一个实用框架，通过物理表征空间中的预测结构来评估动力学保真度。专家定义表征，而参考轨迹决定哪些关系具有预测性并被保留为评估测试。我们在大气预报中演示了该方法，使用ERA5的行星波活动和北环状模演变的表征，并评估了Pangu-Weather、GraphCast和FengWu。这些模型表现出与参考预测结构的明显偏离，而这些偏离并未被传统的预报误差所反映。该框架因此将领域专家的表征转化为对学习到的物理动力学的系统性测试，而无需预先规定这些关系。
英文摘要
Machine-learning models for physical systems are currently evaluated primarily through errors between predicted and reference states and, increasingly, through tests of physical consistency. These metrics assess whether predictions are accurate and satisfy selected physical requirements, but provide limited insight into whether learned trajectories reproduce the underlying dynamics. Domain experts examine such relationships through physical representations that expose relevant processes, interactions, and responses, but these analyses are often separated from typical machine-learning evaluation. We introduce a practical framework for evaluating dynamical fidelity through predictive structure in physical representation spaces. Experts define the representations, while reference trajectories determine which relationships are predictive and retained as evaluation tests. We demonstrate the approach in atmospheric forecasting using ERA5 representations of planetary-wave activity and Northern Annular Mode evolution, and evaluate Pangu-Weather, GraphCast, and FengWu. The models exhibit distinct departures from reference predictive structure that are not reflected by conventional forecast errors. The framework thereby turns domain-expert representations into systematic tests of learned physical dynamics without prescribing the relationships in advance.
