arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 1902.03525stat.ME
收藏

BOLT-SSI:一种筛选超高维数据交互效应的统计方法

BOLT-SSI: A Statistical Approach to Screening Interaction Effects for Ultra-High Dimensional Data

Min Zhou, Mingwei Dai, Yuan Yao, Jin Liu, Can Yang, Heng Peng

摘要:

在各种应用中,检测预测因子对响应变量的交互作用是至关重要的一步。在本文中,我们首先提出了一个简单的方法,确保筛选相互作用(SSI)。尽管其计算复杂度为$O(p^2n)$,但SSI对于中等维度的问题(例如,$p=10^3\sim10^4$),没有遗传假设。到超高维问题(例如,$p = 10^6$)的基础上,基于离散化相关的布尔表示和运算以及离散变量列联表,提出了一种快速算法BOLT-SSI。建立了SSI和BOLT-SSI的统计理论,保证了其可靠的筛选性能。SSI和BOLT-SSI的性能进行了全面的模拟和真实的案例研究。数值结果表明,SSI和BOLT-SSI往往可以优于他们的竞争对手在计算效率和统计精度。所提出的方法可以应用于完全检测与超过30万个预测因子的相互作用。基于这项研究,我们认为有必要重新思考统计准确性和计算效率之间的关系。我们已经表明,统计方法的计算性能往往可以大大提高,通过探索计算架构的优势与可容忍的统计精度损失。

英文摘要:

Detecting interaction effects among predictors on the response variable is a crucial step in various applications. In this paper, we first propose a simple method for sure screening interactions (SSI). Although its computation complexity is $O(p^2n)$, SSI works well for problems of moderate dimensionality (e.g., $p=10^3\sim10^4$), without the heredity assumption. To ultra-high dimensional problems (e.g., $p = 10^6$), motivated by discretization associated Boolean representation and operations and the contingency table for discrete variables, we propose a fast algorithm, named "BOLT-SSI". The statistical theory has been established for SSI and BOLT-SSI, guaranteeing their sure screening property. The performance of SSI and BOLT-SSI are evaluated by comprehensive simulation and real case studies. Numerical results demonstrate that SSI and BOLT-SSI can often outperform their competitors in terms of computational efficiency and statistical accuracy. The proposed method can be applied for fully detecting interactions with more than 300,000 predictors. Based on this study, we believe that there is a great need to rethink the relationship between statistical accuracy and computational efficiency. We have shown that the computational performance of a statistical method can often be greatly improved by exploring the advantages of computational architecture with a tolerable loss of statistical accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑