arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.05931cs.LGcs.AIstat.ML
收藏

通过减少数据访问时间加快学习速度

Faster Learning by Reduction of Data Access Time

Panjab University(潘贾布大学)

Vinod Kumar Chauhan, Anuj Sharma, Kalpana Dahiya

摘要:

如今,机器学习的主要挑战是大数据挑战。大数据问题由于大量的数据点或每个数据点中的大量特征,或两者兼而有之,模型的训练变得非常缓慢。训练时间有两个主要组成部分:访问数据的时间和处理数据的时间。到目前为止,研究只集中在第二部分,即,从数据中学习在本文中,我们提出了一种可能的解决方案来处理大数据问题的机器学习。我们的想法是通过提出系统采样和循环/顺序采样来减少数据访问时间,以减少训练时间,从而从数据集中选择小批量。为了证明所提出的采样技术的有效性,我们使用了经验风险最小化,这是常用的机器学习问题,强凸和光滑的情况下。该问题已解决SAG,SAGA,SVRG,SAAG-II和MBSGD(小批量SGD),每一个使用两步确定技术,即恒定步长和回溯线搜索方法。理论结果证明了系统抽样、循环抽样和广泛应用的随机抽样技术的收敛性,并达到了预期的效果。使用基准数据集的实验结果证明了所提出的采样技术的有效性,并显示出高达六倍的训练速度。

英文摘要:

Nowadays, the major challenge in machine learning is the Big Data challenge. The big data problems due to large number of data points or large number of features in each data point, or both, the training of models have become very slow. The training time has two major components: Time to access the data and time to process (learn from) the data. So far, the research has focused only on the second part, i.e., learning from the data. In this paper, we have proposed one possible solution to handle the big data problems in machine learning. The idea is to reduce the training time through reducing data access time by proposing systematic sampling and cyclic/sequential sampling to select mini-batches from the dataset. To prove the effectiveness of proposed sampling techniques, we have used Empirical Risk Minimization, which is commonly used machine learning problem, for strongly convex and smooth case. The problem has been solved using SAG, SAGA, SVRG, SAAG-II and MBSGD (Mini-batched SGD), each using two step determination techniques, namely, constant step size and backtracking line search method. Theoretical results prove the same convergence for systematic sampling, cyclic sampling and the widely used random sampling technique, in expectation. Experimental results with bench marked datasets prove the efficacy of the proposed sampling techniques and show up to six times faster training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑