Technion-IIT(以色列理工学院);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Tel Aviv University(特拉维夫大学);Institute for Advanced Study(高等研究院)
Shay Moran, Amir Shpilka, Avi Wigderson, Amir Yehudayoff
备注:
The final version is due to be published in the collection of papers "A Journey through Discrete Mathematics. A Tribute to Jiri Matousek" edited by Martin Loebl, Jaroslav Nesetril and Robin Thomas, due to be published by Springer
In this work we study the quantitative relation between VC-dimension and two other basic parameters related to learning and teaching. Namely, the quality of sample compression schemes and of teaching sets for classes of low VC-dimension. Let $C$ be a binary concept class of size $m$ and VC-dimension $d$. Prior to this work, the best known upper bounds for both parameters were $\log(m)$, while the best lower bounds are linear in $d$. We present significantly better upper bounds on both as follows. Set $k = O(d 2^d \log \log |C|)$.
We show that there always exists a concept $c$ in $C$ with a teaching set (i.e. a list of $c$-labeled examples uniquely identifying $c$ in $C$) of size $k$. This problem was studied by Kuhlmann (1999). Our construction implies that the recursive teaching (RT) dimension of $C$ is at most $k$ as well. The RT-dimension was suggested by Zilles et al. and Doliwa et al. (2010). The same notion (under the name partial-ID width) was independently studied by Wigderson and Yehudayoff (2013). An upper bound on this parameter that depends only on $d$ is known just for the very simple case $d=1$, and is open even for $d=2$. We also make small progress towards this seemingly modest goal.
We further construct sample compression schemes of size $k$ for $C$, with additional information of $k \log(k)$ bits. Roughly speaking, given any list of $C$-labelled examples of arbitrary length, we can retain only $k$ labeled examples in a way that allows to recover the labels of all others examples in the list, using additional $k\log (k)$ information bits. This problem was first suggested by Littlestone and Warmuth (1986).
Quadratic Upper Bound for Recursive Teaching Dimension of Finite VC Classes
Tsinghua University(清华大学);Peking University(北京大学)
本文研究了有限规模概念类的递归教学维数(RTD)与VC维数(VCD)之间的定量关系。概念类$\mathcal C \subseteq \{0, 1\}^n$的RTD由Zilles等人(2011)提出,是一种组合复杂性度量,其特征在于根据递归教学模型识别$\mathcal C$中的概念所需的最坏情况示例数。对于任何具有$\mathrm{VCD}(\mathcal C)=d$的有限概念类$\mathcal C \subseteq \{0,1\}^n$,Simon & Zilles(2015)提出了一个开放问题$\mathrm{RTD}(\mathcal C) = O(d)$,即,RTD的上界是由VCD决定的吗?以前,最著名的结果是Chen等人(2016)的指数上限$\mathrm{RTD}(\mathcal C) = O(d \cdot 2^d)$。在本文中,我们展示了一个二次上限:$\mathrm{RTD}(\mathcal C) = O(d^2)$,更接近于一个开放的问题的答案。我们还讨论了充分解决问题的挑战。
Department of Computer Science, University of British Columbia(英属哥伦比亚大学计算机科学系);Department of Mathematics, Ruhr-University Bochum(波鸿鲁尔大学数学系);Department of Computer Science, University of Regina(里贾纳大学计算机科学系)
What Makes a Good Example? Modeling Exemplar Selection with Neural Network Representations
Department of Psychology, University of Southern California(心理学系,南加州大学);Department of Statistics and Data Science, University of California, Los Angeles(统计与数据科学系,加州大学洛杉矶分校)
Spectral Pruning: Compressing Deep Neural Networks via Spectral Analysis and its Generalization Error
The University of Tokyo(东京大学);RIKEN(日本理化学研究所);iPride Co., Ltd.(iPride株式会社);NTT DATA Mathematical Systems Inc.(NTT数据数学系统公司);NTT Data Corporation(NTT数据公司)
Compressing deep neural networks by matrix product operators
Renmin University of China(中国人民大学);Institute of Physics, Chinese Academy of Sciences(中国科学院物理研究所);Peng Cheng Laboratory(鹏城实验室);University of Chinese Academy of Sciences(中国科学院大学);RIKEN Brain Science Institute(日本理化学研究所脑科学研究所)
IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems
Intrinsic Lab, Centre for Sensors, Instrumentation and Cyber-Physical System Engineering (Centre for SeNSE), Indian Institute of Technology Delhi(印度理工学院德里分校传感器、仪器仪表和网络物理系统工程中心(传感中心)本征实验室);RSL Quantum, FITT, IIT Delhi(印度理工学院德里分校 FITT 学院 RSL 量子实验室)