牛顿-拉夫森方法在序列数据预测中的高效实现
Efficient Implementation Of Newton-Raphson Methods For Sequential Data Prediction
AI总结:
本文提出一种高效实现牛顿-拉夫森方法的算法,将计算复杂度从二次降至线性,同时保持二次方法的优势,适用于大数据应用。
摘要:
我们研究了现实生活中大数据应用中序列线性数据预测的问题。第二阶算法,即牛顿-拉夫森方法,相较于第一阶算法(如在线梯度下降),在渐近意义上以更快速度达到最佳可能的线性数据预测性能。然而,这些方法的实现通常在大数据应用中不可行,因为计算需求极高。常规实现的牛顿-拉夫森方法需要计算复杂度为O(M²)(对于M维特征向量),而第一阶算法只需O(M)。为此,本文提出一种高效实现方法,将牛顿-拉夫森方法的计算复杂度从二次降至线性规模。所提出的算法保留了二次方法的优点,同时提供O(M)的计算复杂度。我们利用连续特征向量的移位性质,不依赖任何统计假设。因此,常规和快速实现在线性意义上均达到相同的均方误差性能。我们在真实生活中序列大数据集上展示了所提算法的计算效率,并证明其在数值上是稳定的。
英文摘要:
We investigate the problem of sequential linear data prediction for real life big data applications. The second order algorithms, i.e., Newton-Raphson Methods, asymptotically achieve the performance of the "best" possible linear data predictor much faster compared to the first order algorithms, e.g., Online Gradient Descent. However, implementation of these methods is not usually feasible in big data applications because of the extremely high computational needs. Regular implementation of the Newton-Raphson Methods requires a computational complexity in the order of $O(M^2)$ for an $M$ dimensional feature vector, while the first order algorithms need only $O(M)$. To this end, in order to eliminate this gap, we introduce a highly efficient implementation reducing the computational complexity of the Newton-Raphson Methods from quadratic to linear scale. The presented algorithm provides the well-known merits of the second order methods while offering the computational complexity of $O(M)$. We utilize the shifted nature of the consecutive feature vectors and do not rely on any statistical assumptions. Therefore, both regular and fast implementations achieve the same performance in the sense of mean square error. We demonstrate the computational efficiency of our algorithm on real life sequential big datasets. We also illustrate that the presented algorithm is numerically stable.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。