arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.05389cs.LGstat.ML
收藏

全预测因子

Omnipredictors

VMware Research(VMware研究院);Microsoft Research(微软研究院);Stanford University(斯坦福大学);USC(南加州大学)

Parikshit Gopalan, Adam Tauman Kalai, Omer Reingold, Vatsal Sharan, Udi Wieder

摘要:

损失最小化是机器学习中的主导范例,其中预测器被训练以最小化取决于不确定事件的某个损失函数(例如,“明天会下雨吗?'').不同的损失函数意味着不同的学习算法,并且有时意味着非常不同的预测器。虽然广泛且有吸引力,但这种方法的明显缺点是在学习时可能不知道损失函数,这要求算法使用最佳猜测损失函数。我们提出了一个严格的新范式损失最小化机器学习的损失函数可以忽略在学习的时候,只有在决定一个动作时考虑。我们引入了一个(${\mathcal{L}},\mathcal{C}$)-omnipredictor的概念,它可以用来优化家庭${\mathcal{L}}$中的任何损失。一旦设置了损失函数,预测器的输出就可以进行后处理(单个预测的简单单变量数据独立转换),以便与$\mathcal{C}$类中的任何假设进行比较。后处理本质上是如果预测器的输出是不确定事件的真实概率,则将执行的操作。在某种意义上,全预测器从类$\mathcal{C}$中提取所有预测能力,而不管$\mathcal{L}$中的损失函数如何。我们表明,这种“损失遗忘”的学习是可行的,通过连接到多校准,一个概念引入的上下文中的算法公平性。此外,我们还展示了如何将多校准视为不可知提升的解决方案概念,从而为过去的结果提供新的启示。最后,我们将我们的见解转移回算法公平性的上下文中,通过提供多组损失最小化的全方位预测。

英文摘要:

Loss minimization is a dominant paradigm in machine learning, where a predictor is trained to minimize some loss function that depends on an uncertain event (e.g., "will it rain tomorrow?''). Different loss functions imply different learning algorithms and, at times, very different predictors. While widespread and appealing, a clear drawback of this approach is that the loss function may not be known at the time of learning, requiring the algorithm to use a best-guess loss function. We suggest a rigorous new paradigm for loss minimization in machine learning where the loss function can be ignored at the time of learning and only be taken into account when deciding an action. We introduce the notion of an (${\mathcal{L}},\mathcal{C}$)-omnipredictor, which could be used to optimize any loss in a family ${\mathcal{L}}$. Once the loss function is set, the outputs of the predictor can be post-processed (a simple univariate data-independent transformation of individual predictions) to do well compared with any hypothesis from the class $\mathcal{C}$. The post processing is essentially what one would perform if the outputs of the predictor were true probabilities of the uncertain events. In a sense, omnipredictors extract all the predictive power from the class $\mathcal{C}$, irrespective of the loss function in $\mathcal{L}$. We show that such "loss-oblivious'' learning is feasible through a connection to multicalibration, a notion introduced in the context of algorithmic fairness. In addition, we show how multicalibration can be viewed as a solution concept for agnostic boosting, shedding new light on past results. Finally, we transfer our insights back to the context of algorithmic fairness by providing omnipredictors for multi-group loss minimization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑