Optimizing the Union of Intersections LASSO ($UoI_{LASSO}$) and Vector Autoregressive ($UoI_{VAR}$) Algorithms for Improved Statistical Estimation at Scale
Arizona State University（亚利桑那州立大学）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Oregon State University（俄勒冈州立大学）
The analysis of scientific data of increasing size and complexity requires statistical machine learning methods that are both interpretable and predictive. Union of Intersections (UoI), a recently developed framework, is a two-step approach that separates model selection and model estimation. A linear regression algorithm based on UoI, $UoI_{LASSO}$, simultaneously achieves low false positives and low false negative feature selection as well as low bias and low variance estimates. Together, these qualities make the results both predictive and interpretable. In this paper, we optimize the $UoI_{LASSO}$ algorithm for single-node execution on NERSC's Cori Knights Landing, a Xeon Phi based supercomputer. We then scale $UoI_{LASSO}$ to execute on cores ranging from 68-278,528 cores on a range of dataset sizes demonstrating the weak and strong scaling of the implementation. We also implement a variant of $UoI_{LASSO}$, $UoI_{VAR}$ for vector autoregressive models, to analyze high dimensional time-series data. We perform single node optimization and multi-node scaling experiments for $UoI_{VAR}$ to demonstrate the effectiveness of the algorithm for weak and strong scaling. Our implementations enable to use estimate the largest VAR model (1000 nodes) we are aware of, and apply it to large neurophysiology data 192 nodes).
Safe Screening With Variational Inequalities and Its Application to LASSO
SAS Institute Inc.（赛仕软件研究所有限公司）；Arizona State University（亚利桑那州立大学）
稀疏学习技术已经被常规地用于特征选择，因为所得到的模型通常具有少量的非零条目。安全筛选是一种提高计算效率的技术，它消除了对于正则化参数的某个值保证具有零系数的特征。安全筛选越来越受到关注，因为1）求解稀疏学习公式通常具有很高的计算成本，特别是当特征数量很大时，以及2）需要尝试几个正则化参数来选择合适的模型。在本文中，我们提出了一种方法称为“Sasvi”（安全筛选变分不等式）。Sasvi利用变分不等式为对偶问题提供了充分必要的最优性条件。Lasso筛选的几种现有方法可以被视为所提出的Sasvi的放松版本，因此Sasvi提供了更强的安全筛选规则。我们进一步研究了Sasvi for Lasso的单调性，基于此，可以为每个特征确定一个确定的移除正则化参数。合成和真实数据集上的实验结果表明，建议的Sasvi Lasso筛选的有效性。
