FAST-HEP：为高能物理及其他领域编译声明式分析工作流
FAST-HEP: Compiling Declarative Analysis Workflows for High-Energy Physics and Beyond
浏览论文内容
中文总结 AI 辅助
FAST-HEP的Flow引擎将声明式工作流语言、编译器与运行时结合，编译生成与后端无关的执行计划，支持模块化替换与溯源记录，为高能物理及其他领域提供透明可重复的工作流基础。
中文摘要 AI 辅助
高能物理分析越来越依赖复杂的软件工作流，其科学生命周期往往超过底层软件生态系统的生命周期。因此，在适应不断发展的分析软件、数据格式和执行环境的同时保持可重复性，仍然是一项重大挑战。这些挑战并非高能物理所独有，而是许多数据密集型科学分析共同面临的问题。我们提出了FAST-HEP及其工作流引擎Flow，它结合了声明式工作流语言、编译器和运行时。Flow将工作流的科学描述与其实现和执行分离开来，并通过归一化、图构建、依赖分析和执行规划将工作流编译为与后端无关的执行计划。随后，通用运行时使用可替换的能力来协调生成的计划。这种架构支持分析操作、执行后端和存储技术的静态验证与模块化替换，同时在编译和执行过程中记录溯源信息。尽管Flow是为满足高能物理分析的需求而开发的，但其工作流模型和协调层与领域无关。通过将编译器技术应用于科学分析工作流，FAST-HEP为透明、可扩展、可移植和可重复的工作流提供了基础，使科学分析及其支持的软件生态系统能够独立演进。
英文摘要
High-energy physics analyses increasingly rely on complex software workflows whose scientific lifetime often exceeds that of the underlying software ecosystem. Maintaining reproducibility while accommodating evolving analysis software, data formats, and execution environments therefore remains a significant challenge. These challenges are not unique to high-energy physics and are shared by many data-intensive scientific analyses. We present FAST-HEP and its workflow engine, Flow, which combines a declarative workflow language, compiler, and runtime. Flow separates the scientific description of a workflow from its implementation and execution, and compiles workflows into backend-independent execution plans through normalization, graph construction, dependency analysis, and execution planning. A common runtime then orchestrates the resulting plan using replaceable capabilities. This architecture enables static validation and modular replacement of analysis operations, execution backends, and storage technologies, while recording provenance throughout compilation and execution. Although developed for the requirements of high-energy physics analysis, Flow's workflow model and orchestration layer are domain-independent. By applying compiler techniques to scientific analysis workflows, FAST-HEP provides a foundation for workflows that are transparent, extensible, portable, and reproducible, allowing scientific analyses and their supporting software ecosystems to evolve independently.
