We present Gaptron, a randomized first-order algorithm for online multiclass classification. In the full information setting we show expected mistake bounds with respect to the logistic loss, hinge loss, and the smooth hinge loss with constant regret, where the expectation is with respect to the learner's randomness. In the bandit classification setting we show that Gaptron is the first linear time algorithm with $O(K\sqrt{T})$ expected regret, where $K$ is the number of classes. Additionally, the expected mistake bound of Gaptron does not depend on the dimension of the feature vector, contrary to previous algorithms with $O(K\sqrt{T})$ regret in the bandit classification setting. We present a new proof technique that exploits the gap between the zero-one loss and surrogate losses rather than exploiting properties such as exp-concavity or mixability, which are traditionally used to prove logarithmic or constant regret bounds.
Consistent Robust Adversarial Prediction for General Multiclass Classification
Carnegie Mellon University(卡内基梅隆大学);Conversant LLC(Conversant有限责任公司);California Institute of Technology(加州理工学院);University of Illinois at Chicago(伊利诺伊大学芝加哥分校);University of Pennsylvania(宾夕法尼亚大学)
Learning with Multiclass AUC: Theory and Algorithms
University of Chinese Academy of Sciences(中国科学院大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);Sun Yat-sen University(中山大学);Peng Cheng Laboratory(鹏城实验室)
Convex Calibration Dimension for Multiclass Loss Matrices
Indian Institute of Science(印度科学学院)
我们研究了一般多类学习问题的代理损失函数的一致性,定义了一个一般的多类损失矩阵。我们扩展的概念,分类校准,这已经研究了二进制和多类0-1分类问题(和某些其他特定的学习问题),一般的多类设置,并推导出必要和充分条件的替代损失校准相对于损失矩阵在这种设置。然后,我们引入了多类损失矩阵的凸校准维度的概念,它测量预测空间的最小“尺寸”,在该预测空间中,可以设计相对于损失矩阵校准的凸代理。我们推导出这个量的上界和下界,并使用这些结果来分析各种损失矩阵。特别是,我们应用我们的框架来研究各种子集的排名损失,并使用凸校准尺寸作为工具来显示这些损失的各种类型的凸校准代理的存在和不存在。我们的研究结果加强了Duchi et al.(2010)和Calauzenes et al.(2012)关于子集排序中不存在某些类型的凸校准代理的最新结果。我们预计凸校准维度可能被证明是一个有用的工具,在研究和设计一般的多类学习问题的代理损失。
Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data
Department of Computer Science University of British Columbia(计算机科学系不列颠哥伦比亚大学);University of British Columbia Canada(不列颠哥伦比亚大学加拿大);CIFAR AI Chair (Amii) &(CIFAR人工智能主席(Amii));Department of Electrical and Computer Engineering University of British Columbia(电气与计算机工程系不列颠哥伦比亚大学)
Beyond Bandit Feedback in Online Multiclass Classification
Università degli Studi di Milano(米兰大学);Sapienza Università di Roma(罗马大学);Dept. of Computer Science(计算机科学系);Dept. of Computer, Control and Management Engineering(计算机、控制与管理工程系);DSRC
School of Computing, National University of Singapore(新加坡国立大学计算机学院);Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, France(法国图卢兹大学ENAC ISAE-SUPAERO ONERA联合体);Institute for Infocomm Research, A*STAR , Singapore(新加坡A*STAR信息通信研究所);IPAL, IRL 2955, Singapore(新加坡IPAL实验室);Department of Mathematics, National University of Singapore(新加坡国立大学数学系)
Work While They Sleep: Exploiting Evaluation Latency for Fully Bayesian Optimization
Vector Institute(向量研究所);INRIA Saclay(法国国家信息与自动化研究所萨克雷研究中心);University of Waterloo(滑铁卢大学);Waterloo Institute for Nanotechnology(滑铁卢纳米技术研究所);Western University(西安大略大学)
Plug-In Classification of Drift Functions in Diffusion Processes Using Neural Networks
Université Paris-Dauphine, PSL Chaire DIALog, Fondation du Risque Institut Louis Bachelier(巴黎-第十大学,PSL DIALog研究中心,风险基金会路易·巴舍利埃研究所);LaMME, University of Paris-Saclay(LaMME,巴黎-萨克雷大学);CEREMADE, CNRS Université Paris-Dauphine, PSL(CEREMADE,国家科学研究中心巴黎-第十大学,PSL)
Exploiting Edge Features for Transferable Adversarial Attacks in Distributed Machine Learning
Department of Excellence in Robotics and AI, TeCIP, Scuola Superiore Sant'Anna(机器人与人工智能卓越部门,TeCIP,圣安娜高等学院);Department of Electrical and Electronic Engineering, University of Cagliari(电气与电子工程系,卡利亚里大学)