arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 0812.2263math.STstat.TH
收藏

通过更高批评阈值进行特征选择:最佳相图

Feature selection by Higher Criticism thresholding: optimal phase diagram

David Donoho, Jiashun Jin

摘要:

我们考虑两类线性分类在一个高维,低样本量设置。只有一小部分特征是有用的,有用的特征对我们来说是未知的,每个有用的特征对分类决策的贡献很小-这种设置被称为罕见/弱模型(RW模型)。我们通过对特征$z$分数进行阈值化来选择特征。阈值由{ \it较高批评}(HC)设置。令$\pee_i$表示与第$i$个$z$分数相关联的$P$值,而$\pee_{(i)}$表示$P$值集合的第$i$个顺序统计量. HC阈值(HCT)是$z$评分的顺序统计量,对应于使$(i/n - \pee_{(i)})/\sqrt{\pee_{(i)}(1-\pee_{(i)})}$最大化的指标$i$。理想阈值优化了分类误差。在\cite{PNAS}中,我们发现HCT在数值上接近理想阈值。我们正式研究RW模型的渐近框架,考虑越来越多的功能和相对较少的观测序列的问题。我们表明,沿着这个序列,理想HCT的限制性能基本上是一样好的理想阈值的限制性能。我们的研究结果描述了二维{ \it相空间},一个二维图的坐标量化的“罕见”和“弱”的RW模型。相空间可以划分为两个区域-一个是理想阈值分类成功的区域,另一个是特征太弱、太稀少以至于它必须失败的区域。令人惊讶的是,理想HCT成功和失败的区域在相图中形成了完全相同的分区。其他阈值方法,例如FDR阈值选择,在相空间的基本上比HCT或理想阈值更小的区域中是成功的。

英文摘要:

We consider two-class linear classification in a high-dimensional, low-sample size setting. Only a small fraction of the features are useful, the useful features are unknown to us, and each useful feature contributes weakly to the classification decision -- this setting was called the rare/weak model (RW Model). We select features by thresholding feature $z$-scores. The threshold is set by {\it higher criticism} (HC). Let $\pee_i$ denote the $P$-value associated to the $i$-th $z$-score and $\pee_{(i)}$ denote the $i$-th order statistic of the collection of $P$-values. The HC threshold (HCT) is the order statistic of the $z$-score corresponding to index $i$ maximizing $(i/n - \pee_{(i)})/\sqrt{\pee_{(i)}(1-\pee_{(i)})}$. The ideal threshold optimizes the classification error. In \cite{PNAS} we showed that HCT was numerically close to the ideal threshold. We formalize an asymptotic framework for studying the RW model, considering a sequence of problems with increasingly many features and relatively fewer observations. We show that along this sequence, the limiting performance of ideal HCT is essentially just as good as the limiting performance of ideal thresholding. Our results describe two-dimensional {\it phase space}, a two-dimensional diagram with coordinates quantifying "rare" and "weak" in the RW model. Phase space can be partitioned into two regions -- one where ideal threshold classification is successful, and one where the features are so weak and so rare that it must fail. Surprisingly, the regions where ideal HCT succeeds and fails make the exact same partition of the phase diagram. Other threshold methods, such as FDR threshold selection, are successful in a substantially smaller region of the phase space than either HCT or Ideal thresholding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑