稀疏分类：可扩展离散优化视角
Sparse Classification: a scalable discrete optimization perspective
AI总结：
本文将稀疏分类建模为二元凸优化问题，提出可精确求解的割平面/外逼近算法，并证明其在大规模稀疏 logistic 回归与 SVM 中兼具支撑恢复、稀疏性和计算可扩展性。
摘要：
我们将具有 n 个样本、p 个特征的稀疏分类问题表述为一个二元凸优化问题，并提出一种割平面算法来精确求解。对于稀疏 logistic 回归和稀疏 SVM，我们的算法可在数分钟内为 n 和 p 达到上万规模的问题找到最优解。在合成数据上，该算法在大样本情形下实现完美的支撑恢复：即存在某个 n_0，当 n<n_0 时，算法需要很长时间才能找到最优解且无法恢复正确支撑；而当 n≥n_0 时，算法能快速检测出所有真实特征，且不会返回任何虚假特征。相比之下，尽管 Lasso 能准确检测所有真实特征，但即使观测数量增加，它仍持续返回错误特征。因此，在大量真实世界实验中，我们的外逼近算法能返回更稀疏的分类器，同时取得与 Lasso 相近的预测精度。为支撑这些观察，我们分析了分类中确保完全支撑恢复所需样本量的条件。在关于数据生成过程的若干假设下，我们证明信息论限制给出 n_0<C(2+σ²)k log(p-k)，其中 C>0 为某个常数。
英文摘要：
We formulate the sparse classification problem of $n$ samples with $p$ features as a binary convex optimization problem and propose a cutting-plane algorithm to solve it exactly. For sparse logistic regression and sparse SVM, our algorithm finds optimal solutions for $n$ and $p$ in the $10,000$s within minutes. On synthetic data our algorithm achieves perfect support recovery in the large sample regime. Namely, there exists a $n_0$ such that the algorithm takes a long time to find the optimal solution and does not recover the correct support for $n<n_0$, while for $n\geqslant n_0$, the algorithm quickly detects all the true features, and does not return any false features. In contrast, while Lasso accurately detects all the true features, it persistently returns incorrect features, even as the number of observations increases. Consequently, on numerous real-world experiments, our outer-approximation algorithms returns sparser classifiers while achieving similar predictive accuracy as Lasso. To support our observations, we analyze conditions on the sample size needed to ensure full support recovery in classification. Under some assumptions on the data generating process, we prove that information-theoretic limitations impose $n_0 < C \left(2 + σ^2\right) k \log(p-k)$, for some constant $C>0$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
