错误指定下的分类：半空间、广义线性模型及其与可演化性的联系
Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Connections to Evolvability
AI总结：
本文针对错误指定下的分类问题，为 Massart 噪声下的半空间学习提出了更简单且能达到 $η+ε$ 精度的恰当学习算法，并将其推广到广义线性模型，同时揭示了与可演化性的联系及 SQ 算法的查询复杂度下界。
摘要：
本文重新审视了错误指定下分类中的若干经典问题。特别地，我们研究了在速率为 $η$ 的 Massart 噪声下学习半空间的问题。在最近的一项工作中，Diakonikolas、Goulekakis 和 Tzamos 通过给出第一个对任意 $ε>0$ 都能学习到精度 $η+ ε$ 的高效算法，解决了一个长期悬而未决的问题。然而，他们的算法输出的是一个复杂的假设，该假设将空间划分为 $\text{poly}(d,1/ε)$ 个区域。在此，我们给出一个简单得多的算法，并在此过程中解决了若干尚未解决的开放问题：\n(1) 我们给出了第一个能够达到 $η+ ε$ 的 Massart 半空间 proper learner（恰当学习器）。我们还给出了多项式时间算法可达到的样本复杂度的改进上界。\n(2) 基于 (1)，我们开发了一个黑盒知识蒸馏过程，用于将任意复杂的分类器转换为一个同样好的 proper classifier（恰当分类器）。\n(3) 通过利用一个简单但被忽视的与可演化性的联系，我们证明任何 SQ 算法都需要超多项式数量的查询才能达到 $\mathsf{OPT} + ε$。\n此外，我们研究了广义线性模型，其中 $\mathbb{E}[Y|\mathbf{X}] = σ(\langle \mathbf{w}^*, \mathbf{X}\rangle)$，$σ$ 为任意奇函数、单调且 Lipschitz 的函数。该族包含前述半空间模型作为特例，但更为丰富，并包含其他基本模型，如 logistic regression（逻辑回归）。我们引入了一个具有挑战性的新 corruption model（损坏模型），它推广了 Massart 噪声，并给出了在该设定下学习的一般算法。我们的算法基于一组在存在错误指定时进行分类学习的核心方法。\n最后，我们对 Massart 噪声下学习半空间的算法进行了实证研究，发现它表现出一些有吸引力的公平性性质。
英文摘要：
In this paper we revisit some classic problems on classification under misspecification. In particular, we study the problem of learning halfspaces under Massart noise with rate $η$. In a recent work, Diakonikolas, Goulekakis, and Tzamos resolved a long-standing problem by giving the first efficient algorithm for learning to accuracy $η+ ε$ for any $ε> 0$. However, their algorithm outputs a complicated hypothesis, which partitions space into $\text{poly}(d,1/ε)$ regions. Here we give a much simpler algorithm and in the process resolve a number of outstanding open questions: (1) We give the first proper learner for Massart halfspaces that achieves $η+ ε$. We also give improved bounds on the sample complexity achievable by polynomial time algorithms. (2) Based on (1), we develop a blackbox knowledge distillation procedure to convert an arbitrarily complex classifier to an equally good proper classifier. (3) By leveraging a simple but overlooked connection to evolvability, we show any SQ algorithm requires super-polynomially many queries to achieve $\mathsf{OPT} + ε$. Moreover we study generalized linear models where $\mathbb{E}[Y|\mathbf{X}] = σ(\langle \mathbf{w}^*, \mathbf{X}\rangle)$ for any odd, monotone, and Lipschitz function $σ$. This family includes the previously mentioned halfspace models as a special case, but is much richer and includes other fundamental models like logistic regression. We introduce a challenging new corruption model that generalizes Massart noise, and give a general algorithm for learning in this setting. Our algorithms are based on a small set of core recipes for learning to classify in the presence of misspecification. Finally we study our algorithm for learning halfspaces under Massart noise empirically and find that it exhibits some appealing fairness properties.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
