Probit regression was first proposed by Bliss in 1934 to study mortality rates of insects. Since then, an extensive body of work has analyzed and used probit or related binary regression methods (such as logistic regression) in numerous applications and fields. This paper provides a fresh angle to such well-established binary regression methods. Concretely, we demonstrate that linearizing the probit model in combination with linear estimators performs on par with state-of-the-art nonlinear regression methods, such as posterior mean or maximum aposteriori estimation, for a broad range of real-world regression problems. We derive exact, closed-form, and nonasymptotic expressions for the mean-squared error of our linearized estimators, which clearly separates them from nonlinear regression methods that are typically difficult to analyze. We showcase the efficacy of our methods and results for a number of synthetic and real-world datasets, which demonstrates that linearized binary regression finds potential use in a variety of inference, estimation, signal processing, and machine learning applications that deal with binary-valued observations or measurements.
Binary Classifier Calibration using an Ensemble of Near Isotonic Regression Models
在数据采掘的许多实际任务中，从数据中学习精确的概率模型是至关重要的。在本文中，我们提出了一种新的非参数校准方法称为\textit{ensemble of near isotonic regression}（ENIR）。该方法可以被认为是BBQ，最近提出的校准方法，以及常用的基于保序回归的校准方法的扩展。ENIR旨在解决保序回归的关键限制，即预测的单调性假设。与BBQ类似，该方法对二元分类器的输出进行后处理以获得校准的概率。因此，它可以与许多现有的分类模型相结合。我们证明了ENIR的性能合成和真实的数据集上常用的二进制分类模型。实验结果表明，该方法优于几种常用的二元分类器标定方法。特别是在真实数据上，ENIR通常在统计学上显著优于其他方法，并且永远不会更差。它能够提高分类器的校准能力，同时保持其区分能力。该方法对于大规模数据集也是计算上易处理的，因为它是$O(N \log N)$时间，其中$N$是样本的数量。
High-Dimensional Regression with Binary Coefficients. Estimating Squared Error and a Phase Transition
We consider a sparse linear regression model Y=Xβ^{*}+W where X has a Gaussian entries, W is the noise vector with mean zero Gaussian entries, and β^{*} is a binary vector with support size (sparsity) k. Using a novel conditional second moment method we obtain a tight up to a multiplicative constant approximation of the optimal squared error \min_β \|Y-Xβ\|_{2}, where the minimization is over all k-sparse binary vectors β. The approximation reveals interesting structural properties of the underlying regression problem. In particular, a) We establish that n^*=2k \log p/ \log (2k/σ^{2}+1) is a p…
Resampled Confidence Regions with Exponential Shrinkage for the Regression Function of Binary Classification
Institute for Computer Science and Control (SZTAKI), Hungarian Research Network (HUN-REN)（计算机科学与控制研究所（SZTAKI）、匈牙利研究网络（HUN-REN））；Dept. of Probability Theory and Statistics, Institute of Mathematics, Eötvös Loránd University (ELTE)（概率论与统计系，数学研究所，埃格德·洛朗德大学（ELTE））
On the Equivalence between Online and Private Learnability beyond Binary Classification
University of Michigan（密歇根大学）
Alon et al. [2019]和Bun et al. [2020]最近表明，在线可学习性和私有PAC可学习性在二进制分类中是等价的。我们研究这种等价性是否扩展到多类分类和回归。首先，我们表明，私人的可学习性意味着在这两种情况下的在线可学习性。我们的扩展涉及到研究一种新的变体的Littlestone尺寸，这取决于一个公差参数和适当的泛化的阈值函数的概念超出二进制分类。第二，我们表明，虽然在线学习能力继续意味着私人学习能力在多类分类，目前的证明技术遇到重大障碍，在回归设置。虽然回归的等价性仍然是开放的，但我们提供了非平凡的充分条件，使在线学习类也可以私下学习。
