arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1502.06187cs.LG
收藏

低VC维的示教与压缩

Teaching and compressing for low VC-dimension

Technion-IIT(以色列理工学院);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Tel Aviv University(特拉维夫大学);Institute for Advanced Study(高等研究院)

Shay Moran, Amir Shpilka, Avi Wigderson, Amir Yehudayoff

摘要:

在这项工作中,我们研究VC-维和其他两个基本参数之间的定量关系的学习和教学。即,样本压缩方案和低VC维类的教学集的质量。令$C$是大小为$m$且VC维为$d$的二进制概念类。在这项工作之前,这两个参数的最佳已知上限是$\log(m)$,而最佳下限是线性的$d$。我们提出了显着更好的上界两者如下。设置$k = O(d 2^d \log \log |C|)$。我们发现,总是存在一个概念$c$在$C$与教学集(即一个列表的$c$标记的例子,唯一地识别$c$在$C$)的大小$k$。Kuhlmann(1999)研究了这个问题。我们的结构意味着,递归教学(RT)的$C$的尺寸是最$k$以及。RT维度由Zilles等人和Doliwa等人(2010)提出。Wigderson和Yehudayoff(2013)独立研究了相同的概念(名称为部分ID宽度)。仅对于非常简单的情况$d=1$已知仅依赖于$d$的该参数的上界,并且甚至对于$d=2$也是开放的。我们也在朝着这个看似不大的目标取得微小进展。我们进一步构建样本压缩方案的大小$k$的$C$,与额外的信息$k \log(k)$位。粗略地说,给定任意长度的$C$标记的示例的任何列表,我们可以以允许使用附加的$k\log (k)$信息比特来恢复列表中所有其他示例的标签的方式仅保留$k$标记的示例.这个问题最早是由Littlestone和Warfall(1986)提出的。

英文摘要:

In this work we study the quantitative relation between VC-dimension and two other basic parameters related to learning and teaching. Namely, the quality of sample compression schemes and of teaching sets for classes of low VC-dimension. Let $C$ be a binary concept class of size $m$ and VC-dimension $d$. Prior to this work, the best known upper bounds for both parameters were $\log(m)$, while the best lower bounds are linear in $d$. We present significantly better upper bounds on both as follows. Set $k = O(d 2^d \log \log |C|)$. We show that there always exists a concept $c$ in $C$ with a teaching set (i.e. a list of $c$-labeled examples uniquely identifying $c$ in $C$) of size $k$. This problem was studied by Kuhlmann (1999). Our construction implies that the recursive teaching (RT) dimension of $C$ is at most $k$ as well. The RT-dimension was suggested by Zilles et al. and Doliwa et al. (2010). The same notion (under the name partial-ID width) was independently studied by Wigderson and Yehudayoff (2013). An upper bound on this parameter that depends only on $d$ is known just for the very simple case $d=1$, and is open even for $d=2$. We also make small progress towards this seemingly modest goal. We further construct sample compression schemes of size $k$ for $C$, with additional information of $k \log(k)$ bits. Roughly speaking, given any list of $C$-labelled examples of arbitrary length, we can retain only $k$ labeled examples in a way that allows to recover the labels of all others examples in the list, using additional $k\log (k)$ information bits. This problem was first suggested by Littlestone and Warmuth (1986).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑