全面的插件式Nexflow工作流执行监控
Comprehensive Plugin-Based Monitoring of Nexflow Workflow Executions
AI总结:
本文提出一种基于插件的Nexflow工作流监控方法,提供更详细灵活的监控能力,兼容wf-instances并支持在线监控。
摘要:
Nextflow是一种广泛用于生物信息学和地球观测等领域的流程管理系统。它通过无环任务序列协调各种工具的分布式数据处理,同时使用容器化(例如Docker)、编排(例如Kubernetes)或批处理(例如SLURM)。对这类工作流执行进行监控具有挑战性,但有助于性能分析、调试和数据溯源。除了Nextflow内置的基本监控功能外,用于创建wf-instances的wf-commons工具在Nextflow社区中被视为标准。我们开发的监控插件提供了一种更详细且灵活的替代方案,兼容wf-instances,同时通过使用Nextflow的插件机制(版本21.10),无需定制Nextflow分支,通过可选的静态构件.jar文件修改而无需重新编译,允许在执行过程中进行在线监控。
英文摘要:
Nextflow is a workflow management system commonly used in fields like bioinformatics and earth observation. It coordinates distributed data processing of various tools as an acyclic sequence of tasks while using, containerization (e.g., Docker), orchestration (e.g., Kubernetes), or batch processing (e.g., SLURM). Monitoring such workflow executions can be challenging but aids performance analysis, debugging, and data provenance. Besides Nexflow's basic built-in monitoring, the wf-commons tool for creating wf-instances is widely regarded as the standard in the Nextflow community. The monitoring plugin we develpoed provides a more detailed and flexible alternative compatible with wf-instances while removing the need for a custom Nextflow fork by using Nextflow's plug-in mechanism (version 21.10), optional direct .jar file changes of static artifacts without recompilation and allows online monitoring during execution.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。