arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.08677cs.LGcs.AI
收藏

GeSS:科学应用中分布偏移下几何深度学习的基准测试

GeSS: Benchmarking Geometric Deep Learning under Scientific Applications with Distribution Shifts

University of Science and Technology of China(中国科学技术大学);Georgia Institute of Technology(佐治亚理工学院);University of California, Santa Barbara(加州大学圣巴巴拉分校)

Deyu Zou, Shikun Liu, Siqi Miao, Victor Fung, Shiyu Chang, Pan Li

AI总结:

GeSS 基准通过覆盖多科学领域和三类分布偏移的 30 种实验设置,系统评估了 3 种 GDL 骨干网络与 11 种学习算法在不同 OOD 信息层级下的表现,为科学场景中的几何深度学习应用提供实证依据。

摘要:

几何深度学习(GDL)因其擅长对具有复杂几何结构的数据进行建模而在科学领域获得了显著关注。然而,很少有工作深入研究其应对分布偏移问题的能力,而分布偏移是许多应用中普遍存在的挑战。为弥补这一空白,我们提出了 GeSS,一个专为在存在分布偏移的科学场景中评估 GDL 模型性能而设计的综合基准。我们的评估数据集涵盖从粒子物理、材料科学到生物化学的多个科学领域,并囊括了条件偏移、协变量偏移和概念偏移等广泛的分布偏移类型。此外,我们研究了从分布外(OOD)测试数据中获取信息的三个层级,包括无 OOD 信息、仅有未标注 OOD 数据,以及带有少量标签的 OOD 数据。总体而言,我们的基准测试包含 30 种不同的实验设置,并在每种设置中评估了 3 种 GDL 骨干网络和 11 种学习算法。我们对评估结果进行了深入分析,旨在为将要使用 GDL 的 GDL 研究者和领域实践者提供见解。

英文摘要:

Geometric deep learning (GDL) has gained significant attention in scientific fields, for its proficiency in modeling data with intricate geometric structures. However, very few works have delved into its capability of tackling the distribution shift problem, a prevalent challenge in many applications. To bridge this gap, we propose GeSS, a comprehensive benchmark designed for evaluating the performance of GDL models in scientific scenarios with distribution shifts. Our evaluation datasets cover diverse scientific domains from particle physics, materials science to biochemistry, and encapsulate a broad spectrum of distribution shifts including conditional, covariate, and concept shifts. Furthermore, we study three levels of information access from the out-of-distribution (OOD) test data, including no OOD information, only unlabeled OOD data, and OOD data with a few labels. Overall, our benchmark results in 30 different experiment settings, and evaluates 3 GDL backbones and 11 learning algorithms in each setting. A thorough analysis of the evaluation results is provided, poised to illuminate insights for GDL researchers and domain practitioners who are to use GDL in their applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑