Understanding and predicting the performance of big data applications running in the cloud or on-premises could help minimise the overall cost of operations and provide opportunities in efforts to identify performance bottlenecks. The complexity of the low-level internals of big data frameworks and the ubiquity of application and workload configuration parameters makes it challenging and expensive to come up with comprehensive performance modelling solutions.
In this paper, instead of focusing on a wide range of configurable parameters, we studied the low-level internals of the MapReduce communication pattern and used a minimal set of performance drivers to develop a set of phase level parametric models for approximating the execution time of a given application on a given cluster. Models can be used to infer the performance of unseen applications and approximate their performance when an arbitrary dataset is used as input. Our approach is validated by running empirical experiments in two setups. On average the error rate in both setups is plus or minus 10% from the measured values.
Comprehensive Benchmarking of Machine Learning Methods for Risk Prediction Modelling from Large-Scale Survival Data: A UK Biobank Study
King’s College London（伦敦国王学院）；British Heart Foundation Centre of Research Excellence（英国心脏基金会卓越研究中心）；School of Cardiovascular & Metabolic Medicine and Sciences, King’s College London（伦敦国王学院心血管与代谢医学与科学学院）；Institute of Cardiovascular Science, University College London（伦敦大学学院心血管科学研究所）
该研究在 UK Biobank 大规模生存数据上基准测试八种线性至深度学习生存模型，发现 COX-PH 稳健，而复杂模型仅在特定样本与预测矩阵条件下占优。
University of Tübingen（图宾根大学）；Technical University of Munich（慕尼黑工业大学）；Helmholtz Centre Geesthacht（亥姆霍兹盖斯特哈赫特研究中心）；Research Center caesar（凯撒研究中心）；Max Planck Institute for Intelligent Systems, Tübingen（马克斯·普朗克智能系统研究所（图宾根））
PyPOTS Research（PyPOTS研究机构）；HKUST（香港科技大学）；King’s College London（伦敦国王学院）；University of Oxford（牛津大学）；Sichuan University（四川大学）；University of Washington（华盛顿大学）；GaTech（佐治亚理工学院）；HKUST(GZ)（香港科技大学（广州））；Northwest University（西北大学）；Squirrel AI（松鼠AI）
XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning
School of Computing and Information Systems, Singapore Management University（计算与信息系统学院，新加坡管理大学）；Institute of Trustworthy Embodied AI, Fudan University（可信具身人工智能研究院，复旦大学）
Plug and Play Bench: Simplifying Big Data Benchmarking Using Containers
最近大数据的蓬勃发展，加上其处理和存储的挑战，引发了分布式数据处理和存储范式的发展，如MapReduce，Spark和NoSQL数据库。随着云计算的出现，在机器集群上处理和存储如此庞大的数据集现在是容易可行的。但是，用户可以依靠有限的工具和方法来衡量和理解本地部署在集群或云中的大数据应用程序的性能。研究人员已经开始通过提供适合大数据应用程序的基准测试套件来探索这一领域。然而，这些工具中的许多是分散的，部署和管理复杂的，并且不提供关于对应用程序进行基准测试的货币成本的透明度。在本文中，我们提出了Plug And Play Bench，这是一种基础设施感知抽象，旨在集成和简化机器集群上大数据基准测试工具的部署。PAPB通过基于底层集群部署框架将工具和设置容器化，自动化安装、配置和执行常见大数据基准工作负载的繁琐过程。我们的概念验证实施利用HiBench作为基准套件，HDP作为集群部署框架，Azure作为云平台。本文进一步说明了基于底层Microsoft Azure云平台的成本指标。
