arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1802.09680cs.LG
收藏

多观测回归

Multi-Observation Regression

University of Colorado, Boulder(科罗拉多大学博尔德分校);University of Victoria(维多利亚大学);Harvard University(哈佛大学);University of Pennsylvania(宾夕法尼亚大学)

Rafael Frongillo, Nishant A. Mehta, Tom Morgan, Bo Waggoner

摘要:

最近的工作介绍了损失函数,它测量基于多个同时观测或结果的预测误差。在本文中,我们将探讨当使用这种多观测损失对数据集的$(x,y)$对回归时所出现的理论和实际问题。当损失仅取决于一个观察时,平均经验损失通过将损失应用于每个对来分解,但是对于多观察情况,经验损失甚至没有被良好地定义,并且如果没有具有完全相同的$x$值的几个$(x,y)$对,统计保证的可能性是不清楚的。我们提出了四个算法正式的经验风险最小化的概念,这个问题,其中两个有统计保证的设置,允许慢和快的收敛速度,但这是由其他两个经验表现。实验结果表明,这些算法在低维环境中的实用性,而下限表现出内在的困难,在高维。最后,我们证明了自然基线,使用传统的单观测损失,通过下界和模拟算法的潜在好处。

英文摘要:

Recent work introduced loss functions which measure the error of a prediction based on multiple simultaneous observations or outcomes. In this paper, we explore the theoretical and practical questions that arise when using such multi-observation losses for regression on data sets of $(x,y)$ pairs. When a loss depends on only one observation, the average empirical loss decomposes by applying the loss to each pair, but for the multi-observation case, empirical loss is not even well-defined, and the possibility of statistical guarantees is unclear without several $(x,y)$ pairs with exactly the same $x$ value. We propose four algorithms formalizing the concept of empirical risk minimization for this problem, two of which have statistical guarantees in settings allowing both slow and fast convergence rates, but which are out-performed empirically by the other two. Empirical results demonstrate practicality of these algorithms in low-dimensional settings, while lower bounds demonstrate intrinsic difficulty in higher dimensions. Finally, we demonstrate the potential benefit of the algorithms over natural baselines that use traditional single-observation losses via both lower bounds and simulations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑