发表机构
University of Notre Dame(圣母大学)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
针对HPC工作流跨站点迁移需手动调整的问题,提出以证据为支撑的HPC站点画像,通过测量、文档提取和试点作业三步自动构建,将站点知识转化为可操作的部署决策,实现工作流预检。
AI 中文摘要
将一个在某个HPC站点开发并测试的工作流迁移到另一个站点,很少能不经一定程度的反复试验就成功。包管理器重建软件环境,容器打包整个文件系统,而诸如backpacks等工作流规范则将工作流连同其软件、数据和资源需求一起打包。这些方法解决了工作流可移植性的一半问题:即工作流需要什么。但没有任何方法描述给定的HPC站点必须如何使用,而这缺失的一半正是即使可移植的工作流在每个新站点也需要手动调整的原因。这一缺口包括站点的资源形态、存储配置、网络权限和操作策略。这些信息可能显式存在于批处理系统中,隐藏在文档的文字里,或深埋在路由器的配置中,使得自动化部署工具难以将站点知识转化为有用的部署决策。我们提出HPC站点画像,这是一种结构化的、以证据为支撑的文档,使这些知识变得可操作。我们通过三个步骤自动构建它,这些步骤与信息所在的位置相对应:测量登录节点,通过有界语言模型智能体从文档中提取类型化字段,以及为符合条件的未解决字段提交试点作业。每个字段都根据其证据进行验证或被丢弃,因此由规则而非模型来决定哪些内容进入画像。该画像随后将工作流预检为执行计划或早期可解释的失败。我们在Purdue Anvil、TACC Stampede3和Notre Dame CRC构建画像,并展示一个对真实工作流进行预检的案例研究。
英文摘要
Moving a workflow developed and tested at one HPC site to another rarely succeeds without some amount of trial and error. Package managers rebuild software environments, containers ship whole filesystems, and workflow specifications such as backpacks package a workflow with its software, data, and resource requirements. These approaches address one half of workflow portability: what a workflow needs. But none describes how a given HPC site must be used, and that missing half is why even a portable workflow requires manual adjustment at each new site. That gap includes the site's resource shape, storage configuration, network permissions, and operating policies. This information may be explicit in the batch system, hidden in the prose of documentation, or buried deep within a router's configuration, making it difficult for an automated deployment tool to turn site knowledge into useful deployment decisions. We propose the HPC site profile, a structured, evidence-backed document that makes this knowledge actionable. We automatically construct it in three steps that mirror where the information lives: measuring the login node, extracting typed fields from documentation with a bounded language-model agent, and submitting pilot jobs for eligible unresolved fields. Every field is verified against its evidence or discarded, so a rule, not the model, decides what enters the profile. The profile then preflights a workflow into an execution plan or an early, explainable failure. We build profiles at Purdue Anvil, TACC Stampede3, and Notre Dame CRC and present a case study of preflighting a real workflow.
CommentsAccepted to the 21st Workshop on Workflows in Support of Large-Scale Science (WORKS 2026), held with SC26, Chicago, IL, USA. 8 pages, 7 figures, 3 tables