AI 中文总结
本文针对HTC环境的工作流组合挑战，提出仿真框架与混合组合策略，实现吞吐量提升、网络开销降低，为分布式系统自动化工作流合成提供可扩展模型。
AI 中文摘要
针对高并发资源效率优化的高通量计算（HTC）环境，需复杂编排来管理跨异构资源的PB级数据。一个关键却常被忽视的挑战是工作流组合：在有向无环图（DAG）内对任务集进行战略分组，以降低执行开销并最大化资源利用率。本文提出一种新型仿真框架，用于表征任务集粒度与系统级约束（如作业延迟、故障率、吞吐量及I/O带宽）之间的相互作用。通过探索高维参数空间，我们量化了不同工作流拓扑的性能敏感性。结果表明，动态平衡任务集独立性与执行分组的混合组合策略，可实现高达3.8倍的吞吐量提升和14.9倍的网络开销降低。我们还提出一种多目标函数，支持策略驱动的优化，使系统架构师能在吞吐量、I/O成本与CPU效率之间的帕累托前沿中进行权衡。这些发现为分布式系统中的自动化工作流合成提供了严谨基础，为下一代科学流水线提供了可扩展模型，所有成果均公开可用。
英文摘要
High-Throughput Computing (HTC) environments tailored for high-concurrency resource efficiency require sophisticated orchestration to manage petabyte-scale data across heterogeneous resources. A critical but often overlooked challenge is workflow composition: the strategic grouping of tasksets within a Directed Acyclic Graph (DAG) to mitigate execution overhead while maximizing resource utilization. This paper presents a novel simulation framework for characterizing the interplay between taskset granularity and system-level constraints (e.g., job latency, failure rate, throughput, and I/O bandwidth). By exploring a high-dimensional parameter space, we quantify the performance sensitivity of diverse workflow topologies. Our results demonstrate that hybrid composition strategies, which dynamically balance taskset independence with execution grouping, can yield up to 3.8x throughput increase and a 14.9x reduction in network overhead. We further propose a multi-metric objective function that enables policy-driven optimization, allowing system architects to navigate the Pareto frontier between throughput, I/O cost, and CPU efficiency. These findings provide a rigorous foundation for automated workflow synthesis in distributed systems, offering a scalable model for next-generation scientific pipelines. All artifacts are publicly available.
Comments10 pages, 13 figures, 3 tables. Extended technical report of paper accepted at IEEE eScience 2026
