arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.03394cs.LGcs.AIcs.MA
收藏

基于Agent的协同随机搜索超参数整定和全局函数优化

Agent-based Collaborative Random Search for Hyper-parameter Tuning and Global Function Optimization

Purdue University(普渡大学)

Ahmad Esmaeili, Zahra Ghorrati, Eric T. Matson

摘要:

超参数优化是训练机器学习模型中最繁琐但最关键的步骤之一。对于这个重要的模型构建阶段,有许多方法,从神谕建议的特定于域的手动调优指南到通用黑盒优化技术的利用。本文提出了一种基于Agent的协作技术,用于为机器学习模型(或一般函数优化问题)中的任意超参数(或决策变量)集找到接近最优的值。所开发的方法形成了一个层次化的基于代理的体系结构,在不同的维度上的搜索操作的分布,并采用了一个合作的搜索过程的基础上,基于自适应宽度的随机抽样技术来定位最优。所提出的模型的行为,特别是对它的设计参数的变化,研究在机器学习和全局函数优化应用程序,其性能进行了比较与两个随机调整策略,在实践中常用的。根据实证结果,所提出的模型在实验分类,回归和多维函数优化任务中优于比较方法,特别是在更高的维度和有限的设备上的计算资源的存在下。

英文摘要:

Hyper-parameter optimization is one of the most tedious yet crucial steps in training machine learning models. There are numerous methods for this vital model-building stage, ranging from domain-specific manual tuning guidelines suggested by the oracles to the utilization of general-purpose black-box optimization techniques. This paper proposes an agent-based collaborative technique for finding near-optimal values for any arbitrary set of hyper-parameters (or decision variables) in a machine learning model (or general function optimization problem). The developed method forms a hierarchical agent-based architecture for the distribution of the searching operations at different dimensions and employs a cooperative searching procedure based on an adaptive width-based random sampling technique to locate the optima. The behavior of the presented model, specifically against the changes in its design parameters, is investigated in both machine learning and global function optimization applications, and its performance is compared with that of two randomized tuning strategies that are commonly used in practice. According to the empirical results, the proposed model outperformed the compared methods in the experimented classification, regression, and multi-dimensional function optimization tasks, notably in a higher number of dimensions and in the presence of limited on-device computational resources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑