本文研究在线线性分类中的自定向学习,首次证明选择预测顺序可在最差顺序与随机顺序之间实现强分离,并给出错误界为 O(d log log n) 或与 n 无关的高效算法。
摘要:
在在线分类中,学习者面对一系列样本,需要以在线方式预测其标签,以最小化总错误次数。在自定向变体中,学习者预先知道样本池,并能自适应地选择预测顺序。本文研究选择预测顺序的能力,并在线性分类这一基本任务上首次建立了最差顺序学习与随机顺序学习之间的强分离。在此之前,这种分离仅在非常受限的概念类中已知,例如一维阈值或轴对齐矩形。我们提出两个主要结果。若 X 是从 d 维单位球面上均匀随机抽取的 n 个点组成的数据集,我们设计了一个高效的自定向学习者,其错误次数为 O(d log log(n)),并能对整个数据集进行分类。若 X 是任意大小为 n 的 d 维数据集,我们设计了一个高效的自定向学习者,能以与 n 无关的错误界预测 X 中 99% 点的标签。相比之下,在最差顺序或随机顺序下,即使点是从单位球面均匀抽取且学习者只需预测其中 1% 点的标签,错误次数也至少为 Ω(d log n)。
英文摘要:
In online classification, a learner is presented with a sequence of examples and aims to predict their labels in an online fashion so as to minimize the total number of mistakes. In the self-directed variant, the learner knows in advance the pool of examples and can adaptively choose the order in which predictions are made. Here we study the power of choosing the prediction order and establish the first strong separation between worst-order and random-order learning for the fundamental task of linear classification. Prior to our work, such a separation was known only for very restricted concept classes, e.g., one-dimensional thresholds or axis-aligned rectangles.
We present two main results. If $X$ is a dataset of $n$ points drawn uniformly at random from the $d$-dimensional unit sphere, we design an efficient self-directed learner that makes $O(d \log \log(n))$ mistakes and classifies the entire dataset. If $X$ is an arbitrary $d$-dimensional dataset of size $n$, we design an efficient self-directed learner that predicts the labels of $99\%$ of the points in $X$ with mistake bound independent of $n$. In contrast, under a worst- or random-ordering, the number of mistakes must be at least $Ω(d \log n)$, even when the points are drawn uniformly from the unit sphere and the learner only needs to predict the labels for $1\%$ of them.
Self-Directed Learning of Convex Labelings on Graphs
Moscow Institute of Physics and Technology (MIPT)(莫斯科物理技术学院);TU Wien(维也纳技术大学);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);CENTAI Institute(CENTAI研究所)
Heterogeneous Directed Hypergraph Neural Network over abstract syntax tree (AST) for Code Classification
School of Computer Science and Technology, East China Normal University(计算机科学与技术学院,东华大学);NPPA Key Laboratory of Publishing Integration Development, ECNUP(出版集成发展关键实验室,ECNUP)
SelfAI: A self-directed framework for long-horizon scientific discovery
University of Electronic Science and Technology of China(电子科技大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Xian Jiaotong University(西安交通大学);Yale University(耶鲁大学);Washington University in St. Louis(圣路易斯华盛顿大学);Monash University(莫纳什大学)
TF-DWGNet: A Directed Weighted Graph Neural Network with Tensor Fusion for Multi-Omics Cancer Subtype Classification
Mathematics and Statistical Science University of Idaho(数学与统计科学大学 Idaho 大学);Computer Science and Engineering Mississippi State University(计算机科学与工程密苏里州立大学)
Cyber Academia-Chemical Engineering (CA-ChemE): A Living Digital Town for Self-Directed Research Evolution and Emergent Scientific Discovery
State Key Laboratory of Heavy Oil Processing, College of Carbon Neutrality Future Technology, China University of Petroleum (Beijing), Beijing 102249, China(石油加工国家重点实验室,碳中和未来技术学院,中国石油大学(北京),北京102249)