呼吁建立基准科学和工程
Call for establishing benchmark science and engineering
摘要:
目前,多学科之间没有一致的基准。甚至没有以前的工作试图将不同类别的基准在多学科。本文考察了基准期限的起源和演变。总结了广泛存在于多学科的度量标准、具有定义属性的标准化数据集、代表性工作负载、代表性数据集和最佳实践等五类基准。我认为,在发展这门学科的过程中,有两个紧迫的挑战:在多个学科之间建立一致的基准测试,以及开发元基准测试来衡量基准测试本身。我建议建立基准科学和工程;主要目标之一是建立一个跨学科的标准基准层次结构。现在是推出多学科基准、标准和评估期刊TBench的适当时机,以传达基准科学和工程的最新技术和实践水平。
英文摘要:
Currently, there is no consistent benchmarking across multi-disciplines. Even no previous work tries to relate different categories of benchmarks in multi-disciplines. This article investigates the origin and evolution of the benchmark term. Five categories of benchmarks are summarized, including measurement standards, standardized data sets with defined properties, representative workloads, representative data sets, and best practices, which widely exist in multi-disciplines. I believe there are two pressing challenges in growing this discipline: establishing consistent benchmarking across multi-disciplines and developing meta-benchmark to measure the benchmarks themselves. I propose establishing benchmark science and engineering; one of the primary goals is to set up a standard benchmark hierarchy across multi-disciplines. It is the right time to launch a multi-disciplinary benchmark, standard, and evaluation journal, TBench, to communicate the state-of-the-art and state-of-the-practice of benchmark science and engineering.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。