arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1609.09481stat.MLcs.LG
收藏

快速学习率和重尾损失

Fast learning rates with heavy-tailed losses

Fred Hutchinson Cancer Research Center(弗雷德·哈钦森癌症研究中心);University of California, Los Angeles(加州大学洛杉矶分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);University of Science, Vietnam(越南理科大学)

Vu Dinh, Lam Si Tung Ho, Duy Nguyen, Binh T. Nguyen

摘要:

我们研究快速学习率时的损失不一定是有界的,可能有一个重尾分布。为了使这样的分析,我们引入了两个新的条件:(i)包络函数$\sup_{f \in \mathcal{F}}|\ell \circ f|$,其中$\ell$是损失函数和$\mathcal{F}$是假设类,存在,是$L^r$ -可积的,和(ii)$\ell$满足多尺度伯恩斯坦的条件$\mathcal{F}$。在这些假设下,我们证明了学习速度快于$O(n^{-1/2})$可以获得,并依赖于$r$和多尺度伯恩斯坦的权力,可以任意接近$O(n^{-1})$。然后,我们验证这些假设,并获得快速学习率的问题,矢量量化的$k$ -均值聚类与重尾分布。这些分析使我们能够获得新的学习率,从理论和实践的角度扩展和补充文献中的现有结果。

英文摘要:

We study fast learning rates when the losses are not necessarily bounded and may have a distribution with heavy tails. To enable such analyses, we introduce two new conditions: (i) the envelope function $\sup_{f \in \mathcal{F}}|\ell \circ f|$, where $\ell$ is the loss function and $\mathcal{F}$ is the hypothesis class, exists and is $L^r$-integrable, and (ii) $\ell$ satisfies the multi-scale Bernstein's condition on $\mathcal{F}$. Under these assumptions, we prove that learning rate faster than $O(n^{-1/2})$ can be obtained and, depending on $r$ and the multi-scale Bernstein's powers, can be arbitrarily close to $O(n^{-1})$. We then verify these assumptions and derive fast learning rates for the problem of vector quantization by $k$-means clustering with heavy-tailed distributions. The analyses enable us to obtain novel learning rates that extend and complement existing results in the literature from both theoretical and practical viewpoints.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑