更短时间约束和早停机制下的AutoML基准测试
AutoML Benchmark with shorter time constraints and early stopping
AI总结：
针对AutoML基准测试（AMLB）现有时间预算较长的问题，本研究引入更短时间约束与早停机制减少计算量，评估11个框架后发现排名一致性较好，但早停会增大性能差异，提升了基准测试的实用性与可及性。
摘要：
自动化机器学习（AutoML）可自动在数据上构建机器学习（ML）模型。评估表格数据新AutoML框架的事实标准是AutoML基准测试（AMLB）。AMLB提出在104个任务上使用1小时和4小时时间预算评估AutoML框架。我们认为基准测试应考虑更短时间约束，因其具有实用价值（如模型需高频重训练时），且能提升AMLB的可及性。本工作考虑两种减少基准测试整体计算量的方式：更小时间约束和早停机制的使用。我们在104个任务上用不同时间约束评估11个AutoML框架，发现框架的相对排名在不同时间约束下相当一致，但早停机制会导致模型性能差异更大。
英文摘要：
Automated Machine Learning (AutoML) automatically builds machine learning (ML) models on data. The de facto standard for evaluating new AutoML frameworks for tabular data is the AutoML Benchmark (AMLB). AMLB proposed to evaluate AutoML frameworks using 1- and 4-hour time budgets across 104 tasks. We argue that shorter time constraints should be considered for the benchmark because of their practical value, such as when models need to be retrained with high frequency, and to make AMLB more accessible. This work considers two ways in which to reduce the overall computation used in the benchmark: smaller time constraints and the use of early stopping. We conduct evaluations of 11 AutoML frameworks on 104 tasks with different time constraints and find the relative ranking of AutoML frameworks is fairly consistent across time constraints, but that using early-stopping leads to a greater variety in model performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
