SAIBench:通过基准测试对科学人工智能进行结构化解释
SAIBench: A Structural Interpretation of AI for Science Through Benchmarks
AI总结:
本文提出结构化解释基准测试方法,通过划分问题空间与度量空间来识别可信运行范围并追溯错误,并在机器学习力场、喷注标记和降水临近预报三个 AI4S 任务中验证了其有效性。
摘要:
科学人工智能(AI4S)是一个新兴的研究领域,它利用机器学习的进展来解决复杂的科学计算问题,旨在提高计算效率和准确性。然而,AI4S 的数据驱动特性缺乏传统科学计算所具有的正确性或准确性保证,这给在现实世界应用中部署 AI4S 模型带来了挑战。为了缓解这些问题,需要更全面的基准测试流程来更好地理解 AI4S 模型。本文介绍了一种新颖的基准测试方法,称为结构化解释,它满足两个关键需求:识别问题空间中的可信运行范围,以及将错误追溯到其计算组件。该方法对问题空间和度量空间进行划分,从而促进对这些空间的结构化探索。通过将结构化解释应用于三种不同的 AI4S 工作负载——机器学习力场(MLFF)、喷注标记和降水临近预报——说明了其实际效用和有效性。这些基准测试有效地对可信运行范围进行了建模,追溯了错误,并揭示了用于改进模型、训练过程和数据采样策略的新视角。这项工作是 SAIBench 项目(一个 AI4S 基准测试套件)的一部分。
英文摘要:
Artificial Intelligence for Science (AI4S) is an emerging research field that utilizes machine learning advancements to tackle complex scientific computational issues, aiming to enhance computational efficiency and accuracy. However, the data-driven nature of AI4S lacks the correctness or accuracy assurances of conventional scientific computing, posing challenges when deploying AI4S models in real-world applications. To mitigate these, more comprehensive benchmarking procedures are needed to better understand AI4S models. This paper introduces a novel benchmarking approach, known as structural interpretation, which addresses two key requirements: identifying the trusted operating range in the problem space and tracing errors back to their computational components. This method partitions both the problem and metric spaces, facilitating a structural exploration of these spaces. The practical utility and effectiveness of structural interpretation are illustrated through its application to three distinct AI4S workloads: machine-learning force fields (MLFF), jet tagging, and precipitation nowcasting. The benchmarks effectively model the trusted operating range, trace errors, and reveal novel perspectives for refining the model, training process, and data sampling strategy. This work is part of the SAIBench project, an AI4S benchmarking suite.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。