arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.06604cs.LGcs.AIcs.MA
收藏

分布式机器学习资源上的混合算法选择与超参数调优:一种分层智能体方法

Hybrid Algorithm Selection and Hyperparameter Tuning on Distributed Machine Learning Resources: A Hierarchical Agent-based Approach

Purdue University(普渡大学)

Ahmad Esmaeili, Julia T. Rayz, Eric T. Matson

AI总结:

本文提出一种全自动协作的分层智能体机制,用于在分布式机器学习资源上同时进行算法选择与超参数调优,理论证明其具有线性时空复杂度,并在24种算法和9个数据集上验证了有效性。

摘要:

算法选择和超参数调优是学术和应用机器学习中的关键步骤。另一方面,由于机器学习资源的数量、多样性和分布式特性的大幅增加,这些步骤正变得日益复杂。将多智能体系统应用于机器学习平台的设计,带来了可扩展性、灵活性和鲁棒性等几项显著特征。本文提出了一种全自动且协作的基于智能体的机制,用于选择分布式组织的机器学习算法,并同时调优其超参数。我们的方法建立在现有的基于智能体的分层机器学习平台之上,并扩展了其查询结构以支持上述功能,且不局限于特定的学习、选择和调优机制。我们进行了理论评估、形式化验证和分析研究,以证明我们技术的正确性、资源利用率和计算效率。结果表明,我们的解决方案是完全正确的,并且相对于可用资源的规模表现出线性的时间和空间复杂度。为了提供具体示例,说明所提出的方法如何能够在一系列算法选项和数据集上有效地适应和执行,我们还使用一个由24种算法和9个数据集组成的系统进行了一系列实验。

英文摘要:

Algorithm selection and hyperparameter tuning are critical steps in both academic and applied machine learning. On the other hand, these steps are becoming ever increasingly delicate due to the extensive rise in the number, diversity, and distributedness of machine learning resources. Multi-agent systems, when applied to the design of machine learning platforms, bring about several distinctive characteristics such as scalability, flexibility, and robustness, just to name a few. This paper proposes a fully automatic and collaborative agent-based mechanism for selecting distributedly organized machine learning algorithms and simultaneously tuning their hyperparameters. Our method builds upon an existing agent-based hierarchical machine-learning platform and augments its query structure to support the aforementioned functionalities without being limited to specific learning, selection, and tuning mechanisms. We have conducted theoretical assessments, formal verification, and analytical study to demonstrate the correctness, resource utilization, and computational efficiency of our technique. According to the results, our solution is totally correct and exhibits linear time and space complexity in relation to the size of available resources. To provide concrete examples of how the proposed methodologies can effectively adapt and perform across a range of algorithmic options and datasets, we have also conducted a series of experiments using a system comprised of 24 algorithms and 9 datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑