我们从理论上分析和比较了以下五种流行的多类分类方法:One vs. All,All Pairs,Tree-based classifiers,带有随机生成代码矩阵的纠错输出代码(ECOC)和MulticlassSVM。在前四种方法中,分类是基于二进制分类的简化。我们考虑的情况下,二元分类器来自一类VC维$d$,特别是从类的半空间在$\reals^d$。我们分析了这些方法的估计误差和近似误差。我们的分析揭示了有趣的结论的实际意义,在各种条件下的不同方法的成功。我们的证明技术采用VC理论的工具来分析假设类的\emph{approximation error}。这与大多数(如果不是全部的话)以前使用VC理论形成鲜明对比,这些理论只处理估计误差。
英文摘要:
We theoretically analyze and compare the following five popular multiclass classification methods: One vs. All, All Pairs, Tree-based classifiers, Error Correcting Output Codes (ECOC) with randomly generated code matrices, and Multiclass SVM. In the first four methods, the classification is based on a reduction to binary classification. We consider the case where the binary classifier comes from a class of VC dimension $d$, and in particular from the class of halfspaces over $\reals^d$. We analyze both the estimation error and the approximation error of these methods. Our analysis reveals interesting conclusions of practical relevance, regarding the success of the different approaches under various conditions. Our proof technique employs tools from VC theory to analyze the \emph{approximation error} of hypothesis classes. This is in sharp contrast to most, if not all, previous uses of VC theory, which only deal with estimation error.
Revisiting Discriminative vs. Generative Classifiers: Theory and Implications
Gaoling School of AI, Renmin University of China(中国人民大学高瓴人工智能学院);School of Software, Shandong University(山东大学软件学院);Tsinghua University(清华大学);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院)
Self-Weighted Robust LDA for Multiclass Classification with Edge Classes
Xi’an Jiaotong University(西安交通大学);Monash University(莫纳什大学);Wenzhou University(温州大学);Qilu University of Technology(齐鲁工业大学);Shandong Academy of Sciences(山东省科学院);Northwestern Polytechnical University(西北工业大学)
Local Binary and Multiclass SVMs Trained on a Quantum Annealer
University of Trento(特伦托大学);Jülich Supercomputing Centre(于利希超级计算中心);University of Iceland(冰岛大学);Trento Institute for Fundamental Physics and Applications(特伦托基础物理与应用研究所);Alma Mater Studiorum - Università di Bologna(博洛尼亚大学);AIDAS(艾达斯)
Is the Statistical Advantage Worth the Cost? An Empirical Comparison of KANs and MLPs for Structured Data Classification
Institute of Computer Science, University of the Philippines Los Baños(菲律宾大学洛斯巴尼奥斯分校计算机科学研究所);Machine Learning and Artificial Intelligence Applications Lab, University of the Philippines Los Baños(菲律宾大学洛斯巴尼奥斯分校机器学习与人工智能应用实验室)
Analysis of an adaptive lead weighted ResNet for multiclass classification of 12-lead ECGs
University of Manchester(曼彻斯特大学);Xi’an Jiaotong University(西安交通大学);University of Exeter(埃克塞特大学);University of Leeds(利兹大学);Loughborough University(拉夫堡大学)
A Comparative Evaluation of Quantification Methods
University of Mannheim(曼海姆大学);RWTH Aachen University(亚琛工业大学);GESIS - Leibniz Institute for the Social Sciences(GESIS莱布尼茨社会科学研究所);Complexity Science Hub(复杂性科学中心);University of Passau(帕绍大学)
Truthful Calibration Errors for Multi-Class Prediction
Peking University(北京大学);Northwestern University(西北大学);Microsoft Research, New England(微软研究院(新英格兰));Northeastern University(东北大学);Khoury College of Computer Sciences(计算机科学学院)
Comparison of machine learning and deep learning techniques in promoter prediction across diverse species
Symbiosis Institute of Technology(共生理工学院);Symbiosis International (Deemed University)(共生国际大学);Symbiosis Center for Applied AI (SCAAI)(共生应用人工智能中心);Symbiosis School of Biological Sciences(共生生物科学学院)