arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.12221cs.SEcs.NI
收藏

挖掘异常数据传输的科学工作流

Mining Scientific Workflows for Anomalous Data Transfers

Huy Tu, George Papadimitriou, Mariam Kiran, Cong Wang, Anirban Mandal, Ewa Deelman, Tim Menzies

摘要:

现代科学工作流是数据驱动的,通常在分布式、异构、高性能计算基础设施上执行。工作流执行中的异常和故障会导致科学生产力的损失和基础设施的低效使用。因此,检测、诊断和缓解这些异常对于可靠和高性能的科学工作流程非常重要。由于这些工作流严重依赖于需要严格QoS约束的高性能网络传输,因此准确检测异常网络性能对于确保可靠和高效的工作流执行至关重要。为了应对这一挑战,我们开发了X-FLASH,一种用于故障TCP工作流传输的网络异常检测工具。X-FLASH结合了新的超参数调整和数据采掘方法,用于提高机器学习算法的性能,以准确地分类异常TCP数据包。X-FLASH利用XGBoost作为集成模型,并将XGBoost与从基于搜索的软件工程中借来的顺序优化器FLASH耦合,以学习最佳模型参数。X-FLASH发现,在不到30次评估中,配置在F测量、G评分和召回方面的表现优于现有方法,相对而言分别高达28%、29%和40%。从(1)大的改进和(2)简单的调优,我们建议未来的研究有额外的调优研究作为一个新的标准,至少在科学的工作流异常检测领域。

英文摘要:

Modern scientific workflows are data-driven and are often executed on distributed, heterogeneous, high-performance computing infrastructures. Anomalies and failures in the workflow execution cause loss of scientific productivity and inefficient use of the infrastructure. Hence, detecting, diagnosing, and mitigating these anomalies are immensely important for reliable and performant scientific workflows. Since these workflows rely heavily on high-performance network transfers that require strict QoS constraints, accurately detecting anomalous network performance is crucial to ensure reliable and efficient workflow execution. To address this challenge, we have developed X-FLASH, a network anomaly detection tool for faulty TCP workflow transfers. X-FLASH incorporates novel hyperparameter tuning and data mining approaches for improving the performance of the machine learning algorithms to accurately classify the anomalous TCP packets. X-FLASH leverages XGBoost as an ensemble model and couples XGBoost with a sequential optimizer, FLASH, borrowed from search-based Software Engineering to learn the optimal model parameters. X-FLASH found configurations that outperformed the existing approach up to 28\%, 29\%, and 40\% relatively for F-measure, G-score, and recall in less than 30 evaluations. From (1) large improvement and (2) simple tuning, we recommend future research to have additional tuning study as a new standard, at least in the area of scientific workflow anomaly detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑