arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1903.00843cs.LGcs.DCstat.ML
收藏

大数据中回归的多元学习

Multiple Learning for Regression in big data

Purdue University(普渡大学)

Xiang Liu, Ziyang Tang, Huyunting Huang, Tonglin Zhang, Baijian Yang

摘要:

具有封闭形式解决方案的回归问题很容易理解,并且当数据集足够小以全部加载到RAM中时可以轻松实现。当数据太大而无法存储在RAM中以计算封闭形式的解决方案时,就会出现挑战。许多技术被提出来克服或减轻内存障碍问题,但解决方案往往是局部最优的。此外,大多数方法需要在更新模型时再次访问原始数据。如果需要同时计算多个模型,也期望并行计算集群。我们提出了多种学习方法,利用一系列足够的统计数据(SS)来解决这一大数据挑战。当计算具有封闭形式解的回归时,这种记忆遗忘方法打破了记忆障碍,包括但不限于线性回归、加权线性回归、具有Box-Cox变换的线性回归(Box-Cox回归)和岭回归模型。SS数组的计算和更新可以在每行级别或每个小批量级别处理。更新模型就像矩阵加减一样简单。此外,不同模型的多个SS阵列可以很容易地同时计算,以在一次通过数据集时获得多个模型。我们在Spark上实现了我们的方法,并在模拟数据集上进行了评估。实验结果表明,该方法可以获得多个模型的封闭解,而训练时间仅为传统方法的一半。

英文摘要:

Regression problems that have closed-form solutions are well understood and can be easily implemented when the dataset is small enough to be all loaded into the RAM. Challenges arise when data is too big to be stored in RAM to compute the closed form solutions. Many techniques were proposed to overcome or alleviate the memory barrier problem but the solutions are often local optimal. In addition, most approaches require accessing the raw data again when updating the models. Parallel computing clusters are also expected if multiple models need to be computed simultaneously. We propose multiple learning approaches that utilize an array of sufficient statistics (SS) to address this big data challenge. This memory oblivious approach breaks the memory barrier when computing regressions with closed-form solutions, including but not limited to linear regression, weighted linear regression, linear regression with Box-Cox transformation (Box-Cox regression) and ridge regression models. The computation and update of the SS array can be handled at per row level or per mini-batch level. And updating a model is as easy as matrix addition and subtraction. Furthermore, multiple SS arrays for different models can be easily computed simultaneously to obtain multiple models at one pass through the dataset. We implemented our approaches on Spark and evaluated over the simulated datasets. Results showed our approaches can achieve closed-form solutions of multiple models at the cost of half training time of the traditional methods for a single model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑