使用分层自适应速率缩放（LARS）优化器评估SystemML中的深度学习
Evaluating Deep Learning in SystemML using Layer-wise Adaptive Rate Scaling(LARS) Optimizer
摘要：
增加深度学习模型的批量大小是一项具有挑战性的任务。虽然它可能有助于在模型的训练阶段充分利用可用的系统内存，但它通常会导致测试准确性的显著损失。LARS通过为深度学习模型的每一层引入自适应学习率来解决这个问题。然而，人们对流行的分布式机器学习系统（如SystemML或MLlib）如何使用此优化器表示怀疑。在这项工作中，我们将LARS优化器应用于使用SystemML实现的深度学习模型。我们使用各种批量大小进行实验，并将LARS优化器的性能与\textit{Stochastic Gradient Descent}进行比较。我们的实验结果表明，LARS优化器的性能显着优于随机梯度下降大批量，即使与分布式机器学习框架，SystemML。
英文摘要：
Increasing the batch size of a deep learning model is a challenging task. Although it might help in utilizing full available system memory during training phase of a model, it results in significant loss of test accuracy most often. LARS solved this issue by introducing an adaptive learning rate for each layer of a deep learning model. However, there are doubts on how popular distributed machine learning systems such as SystemML or MLlib will perform with this optimizer. In this work, we apply LARS optimizer to a deep learning model implemented using SystemML.We perform experiments with various batch sizes and compare the performance of LARS optimizer with \textit{Stochastic Gradient Descent}. Our experimental results show that LARS optimizer performs significantly better than Stochastic Gradient Descent for large batch sizes even with the distributed machine learning framework, SystemML.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
