arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1406.3781cs.LGstat.ML
收藏

从随机可混性到快速率

From Stochastic Mixability to Fast Rates

Australian National University(澳大利亚国立大学);NICTA(澳大利亚国家信息与通信技术局)

Nishant A. Mehta, Robert C. Williamson

摘要:

经验风险最小化(ERM)是统计学习问题的基本学习规则,其中数据根据某些未知分布$\mathsf{P}$生成,并返回从具有小损失$\ell$的固定类$\mathcal{F}$中选择的假设$f$。在参数设置中,根据$(\ell, \mathcal{F},\mathsf{P})$,ERM可以有慢$(1/\sqrt{n})$或快$(1/n)$的过度风险收敛率,作为样本量$n$的函数。有几个结果给出了快速率的充分条件的联合性质的$\ell$,$\mathcal{F}$,和$\mathsf{P}$,如边际条件和伯恩斯坦条件。在具有专家建议设置的非统计预测中,存在类似的慢速率和快速率现象,并且其完全根据损失$\ell$的可混合性来表征(其中对于$\mathcal{F}$或$\mathsf{P}$没有作用)。随机混合性的概念在这两种学习模型之间建立了一座桥梁,在特殊情况下可以简化为经典混合性。本文利用$(\ell,\mathcal{F}, \mathsf{P})$的随机可混性给出了ERM快速率的直接证明,从而为快速率现象提供了新的认识。证明利用了一个旧的结果肯珀曼的解决方案的一般时刻的问题。我们还展示了一个部分的逆,这表明快速速率的ERM随机混合性的表征是可能的。

英文摘要:

Empirical risk minimization (ERM) is a fundamental learning rule for statistical learning problems where the data is generated according to some unknown distribution $\mathsf{P}$ and returns a hypothesis $f$ chosen from a fixed class $\mathcal{F}$ with small loss $\ell$. In the parametric setting, depending upon $(\ell, \mathcal{F},\mathsf{P})$ ERM can have slow $(1/\sqrt{n})$ or fast $(1/n)$ rates of convergence of the excess risk as a function of the sample size $n$. There exist several results that give sufficient conditions for fast rates in terms of joint properties of $\ell$, $\mathcal{F}$, and $\mathsf{P}$, such as the margin condition and the Bernstein condition. In the non-statistical prediction with expert advice setting, there is an analogous slow and fast rate phenomenon, and it is entirely characterized in terms of the mixability of the loss $\ell$ (there being no role there for $\mathcal{F}$ or $\mathsf{P}$). The notion of stochastic mixability builds a bridge between these two models of learning, reducing to classical mixability in a special case. The present paper presents a direct proof of fast rates for ERM in terms of stochastic mixability of $(\ell,\mathcal{F}, \mathsf{P})$, and in so doing provides new insight into the fast-rates phenomenon. The proof exploits an old result of Kemperman on the solution to the general moment problem. We also show a partial converse that suggests a characterization of fast rates for ERM in terms of stochastic mixability is possible.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑