arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.12992cs.LG
收藏

面向神经架构搜索的加性正则化调度策略

Additive regularization schedule for neural architecture search

Mark Potanin, Kirill Vayser, Vadim Strijov

AI总结:

本文针对神经网络结构优化问题,提出加性正则化调度的神经架构搜索方法,通过迭代调整正则项集合优化结构,实验验证该方法可得到低复杂度高精度的高效网络。

摘要:

神经网络结构对预测的准确性与稳定性有着至关重要的影响。神经架构搜索流程可依据代表一组质量准则的损失函数,辅助设计最优神经网络。本文研究神经网络结构优化问题,提出了一种构建包含一组加性元素的损失函数的方法,每个元素被称为正则项,对应神经网络结构的某一部分,代表一项待优化的准则。优化过程以迭代方式调整结构,为优化结构的不同部分,该流程会按照某种调度方案调整正则项集合。作者提出了一种构建加性正则化调度的方法。通过在一组数据集上对比正则化模型与非正则化模型,计算实验表明,所提方法能够找到高效的神经网络结构,生成复杂度低且精度高的网络。

英文摘要:

Neural network structures have a critical impact on the accuracy and stability of forecasting. Neural architecture search procedures help design an optimal neural network according to some loss function, which represents a set of quality criteria. This paper investigates the problem of neural network structure optimization. It proposes a way to construct a loss function, which contains a set of additive elements. Each element is called the regularizer. It corresponds to some part of the neural network structure and represents a criterion to optimize. The optimization procedure changes the structure in iterations. To optimize various parts of the structure, the procedure changes the set of regularizers according to some schedule. The authors propose a way to construct the additive regularization schedule. By comparing regularized models with non-regularized ones for a collection of datasets the computational experiments show that the proposed method finds efficient neural network structure and delivers accurate networks of low complexity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑