arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.14260stat.ME
收藏

多类分类中的新回归视角

A New Regression Lens on Multi-Class Classification

University of Toronto(多伦多大学);Cornell University(康奈尔大学)

Xin Bing, Bingqing Li, Marten Wegkamp

AI总结:

本文通过建立判别方向与回归系数之间的关系,提出基于回归的多类分类新框架,支持结构化、正则化及非参数回归方法,并为l1正则化和降维回归提供理论保证。

摘要:

线性判别分析(LDA)是分类的基本方法,其简单的线性结构有助于解释,并自然适用于多类设置。LDA与几种经典多元技术紧密相关,包括Fisher判别分析、相关分析和线性回归。本文通过建立LDA与多元响应回归之间的显式关系,提出一种新的基于回归的多类分类框架,该框架能够容纳结构化、正则化甚至非参数回归方法。与现有基于回归的方法相比,我们的方法特别适合理论分析:我们开发了一种通用策略,用于推导所提出分类器在所有此类回归过程中的超额误分类风险界限。作为具体应用,我们为两种广泛使用的技术——l1正则化和降维回归——提供了完整的理论保证,这两种方法此前在LDA背景下尚未被完全分析。理论结果通过广泛的模拟研究和真实数据的实证评估得到支持。

英文摘要:

Linear Discriminant Analysis (LDA) is a fundamental method for classification. Its simple linear structure facilitates interpretation, and it is naturally suited to multi-class settings. LDA is also closely connected to several classical multivariate techniques, including Fisher's discriminant analysis, canonical correlation analysis, and linear regression. In this paper, we strengthen the connection between LDA and multivariate response regression by establishing an explicit relationship between discriminant directions and regression coefficients. This characterization yields a new regression-based framework for multi-class classification that accommodates structured, regularized, and even non-parametric regression methods. In contrast to existing regression-based approaches, our formulation is particularly amenable to theoretical analysis: we develop a general strategy for deriving bounds on the excess misclassification risk of the proposed classifier across all such regression procedures. As concrete applications, we provide complete theoretical guarantees for two widely used methods -- $\ell_1$-regularization and reduced-rank regression -- neither of which has previously been fully analyzed in the LDA context. The theoretical results are supported by extensive simulation studies and empirical evaluations on real data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑