最小二乘问题的OEM
OEM for least squares problems
摘要：
我们提出了一种算法，称为OEM（a.k.a.正交化EM），旨在用于各种最小二乘问题。第一步称为主动正交化，通过精心添加更多行来对任意回归矩阵进行正交化。第二步估算新行的响应。第三步解决完整正交设计的最小二乘问题。第二步和第三步有简单的封闭形式，并一直到收敛。该算法适用于普通最小二乘和正则化最小二乘与套索，SCAD，MCP和其他惩罚。它有几个吸引人的理论性质。对于具有奇异回归矩阵的普通最小二乘估计，OEM序列收敛于Moore-Penrose广义逆最小二乘估计。对于SCAD和MCP，OEM序列可以在对固定或发散数量的变量进行足够的迭代之后实现oracle属性。对于具有各种惩罚的普通和正则化最小二乘，OEM序列收敛到具有完全混叠回归矩阵的分组一致性的点。算法的收敛性和收敛速度。这些收敛速度的结果表明，对于相同的数据集，OEM收敛更快的正则化最小二乘比普通最小二乘。这为这些方法提供了一个新的理论比较。数值例子来说明所提出的算法。
英文摘要：
We propose an algorithm, called OEM (a.k.a. orthogonalizing EM), intended for var- ious least squares problems. The first step, named active orthogonization, orthogonalizes an arbi- trary regression matrix by elaborately adding more rows. The second step imputes the responses of the new rows. The third step solves the least squares problem of interest for the complete orthog- onal design. The second and third steps have simple closed forms, and iterate until convergence. The algorithm works for ordinary least squares and regularized least squares with the lasso, SCAD, MCP and other penalties. It has several attractive theoretical properties. For the ordinary least squares with a singular regression matrix, an OEM sequence converges to the Moore-Penrose gen- eralized inverse-based least squares estimator. For the SCAD and MCP, an OEM sequence can achieve the oracle property after sufficient iterations for a fixed or diverging number of variables. For ordinary and regularized least squares with various penalties, an OEM sequence converges to a point having grouping coherence for fully aliased regression matrices. Convergence and convergence rate of the algorithm are examined. These convergence rate results show that for the same data set, OEM converges faster for regularized least squares than ordinary least squares. This provides a new theoretical comparison between these methods. Numerical examples are provided to illustrate the proposed algorithm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
