DiSTILL: 一种用于可重复空间转录组学分析的混合云-HPC工作流系统
DiSTILL: A Hybrid Cloud-HPC Workflow System for Reproducible Spatial Transcriptomics Analysis
AI总结:
提出DiSTILL混合云-HPC工作流系统,通过API后端、Web前端和Python管道生成器,实现空间转录组学分析的可重复执行、队列编排和配置管理,并以IBD分析为例验证系统。
摘要:
空间转录组学工作流越来越多地结合大型注释数据对象、基于笔记本的分析以及必须在高性能计算(HPC)系统上执行的资源密集型统计模型。在实践中,这些工作流通常难以重现,因为配置、验证、阶段执行和工件处理分散在$\textit{ad hoc}$脚本和手动编辑的笔记本中。我们提出了$\textit{DiSTILL}$(通过可解释的潜在学习从空间转录组学进行疾病诊断),一种用于可重复空间转录组(ST)分析的混合云-HPC工作流系统。DiSTILL结合了用$\texttt{FastAPI}$构建的应用程序编程接口(API)后端、Web前端、数据集和预设注册表,以及一个生成运行特定执行包和$\texttt{SLURM}$提交脚本的Python管道生成器。该系统支持本地、安全外壳(SSH)中介和基于拉取的轮询执行模式,从而在持久API启动自动化受限的环境中实现HPC提交。我们通过炎症性肠病(IBD)ST工作流来描述该系统,该工作流将Tan$\textit{et al.}$的分析管道操作化为可审计的应用层。因此,本文的贡献是一个工作流系统贡献,侧重于可重复执行、基于队列的编排、配置语义以及跨分离的云-HPC架构的部署。DiSTILL更广泛的应用目标是支持满足封装分析管道模式假设的用户提供的数据集。
英文摘要:
Spatial transcriptomics workflows increasingly combine large annotated data objects, notebook-based analyses, and resource-intensive statistical models that must be executed on high-performance computing (HPC) systems. In practice, these workflows are often difficult to reproduce because configuration, validation, stage execution, and artifact handling are fragmented across $\textit{ad hoc}$ scripts and manually edited notebooks. We present $\textit{DiSTILL}$ (Disease Diagnosis from Spatial Transcriptomics via Interpretable Latent Learning), a hybrid cloud$-$HPC workflow system for reproducible spatial transcriptomics (ST) analysis. DiSTILL combines an application programming interface (API) backend built with $\texttt{FastAPI}$, a web frontend, a dataset and preset registry, and a Python pipeline generator that materializes run-specific execution bundles and $\texttt{SLURM}$ submission scripts. The system supports local, Secure Shell (SSH)-mediated, and pull-based poller execution modes, enabling HPC submission in environments where persistent API-initiated automation is restricted. We describe the system through the lens of an inflammatory bowel disease (IBD) ST workflow that operationalizes the analytical pipeline of Tan $\textit{et al.}$ into an auditable application layer. Accordingly, the contribution of this paper is a workflow systems contribution centered on reproducible execution, queue-based orchestration, configuration semantics, and deployment across a split cloud$-$HPC architecture. The broader application goal of DiSTILL is to support user-supplied datasets that satisfy the schema assumptions of the wrapped analytical pipeline.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。