基于集成方法的深度学习超参数优化：在质子结构中的应用
Hyperparameter Optimisation in Deep Learning from Ensemble Methods: Applications to Proton Structure
AI总结：
本文提出基于模型集成统计估计器的深度学习超参数自动优化方法，并通过 GPU 并行训练在 NNPDF 质子结构分析中验证其稳健性，实现显著加速与节能。
摘要：
深度学习模型由大量超参数定义，例如网络架构和优化器设置。这些超参数必须与网络权重等模型参数分开确定，并且通常通过临时方法或对结果的人工检查来固定。对超参数进行算法化、客观的确定要求引入专门的目标度量，这些度量不同于模型训练所采用的度量。在此，我们提出一种自动确定深度学习模型中超参数的新方法，该方法基于由对模型空间中底层概率分布进行采样的模型集成所构造的统计估计器。该策略需要同时并行训练多达数百个模型，并且可以通过部署 GPU 等硬件加速器来有效实现。作为概念验证，我们将该方法应用于 NNPDF 框架内质子部分子结构的确定，并证明了所得模型不确定性估计的稳健性。新的 GPU 优化 NNPDF 代码与基于 CPU 的顺序模型训练相比，实现了最高达两个数量级的加速、内存需求的稳定化以及最高达 90% 的能耗降低。虽然聚焦于质子结构，但我们的方法具有完全通用性，适用于任何依赖模型集成超参数优化的深度学习问题。
英文摘要：
Deep learning models are defined in terms of a large number of hyperparameters, such as network architectures and optimiser settings. These hyperparameters must be determined separately from the model parameters such as network weights, and are often fixed by ad-hoc methods or by manual inspection of the results. An algorithmic, objective determination of hyperparameters demands the introduction of dedicated target metrics, different from those adopted for the model training. Here we present a new approach to the automated determination of hyperparameters in deep learning models based on statistical estimators constructed from an ensemble of models sampling the underlying probability distribution in model space. This strategy requires the simultaneous parallel training of up to several hundreds of models and can be effectively implemented by deploying hardware accelerators such as GPUs. As a proof-of-concept, we apply this method to the determination of the partonic substructure of the proton within the NNPDF framework and demonstrate the robustness of the resultant model uncertainty estimates. The new GPU-optimised NNPDF code results in a speed-up of up to two orders of magnitude, a stabilisation of the memory requirements, and a reduction in energy consumption of up to 90% as compared to sequential CPU-based model training. While focusing on proton structure, our method is fully general and is applicable to any deep learning problem relying on hyperparameter optimisation for an ensemble of models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
