arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.09157cs.LG
收藏

基于KL散度的鲁棒满意模型研究

On the KL-Divergence-based Robust Satisficing Model

Fudan University(复旦大学);Shanghai University of Finance and Economics(上海财经大学)

Haojie Yan, Minglong Zhou, Jiayi Guo

AI总结:

针对经验风险最小化受优化器诅咒影响的问题,研究通用损失下基于KL散度的鲁棒满意模型,提出高效求解等方案,实验验证其性能优于现有基准。

摘要:

经验风险最小化是机器学习的基石,但常因经验数据分布与真实数据生成分布之间的差异导致的“优化器诅咒”而受阻。为应对这一挑战,鲁棒满意框架近年被提出,用于缓解真实分布的不确定性,该方法凭借可解释的超参数与更强的性能保证,受到学术界越来越多的关注。然而,由于在通用损失函数下高效求解该模型存在计算挑战,其在通用机器学习问题(尤其是深度神经网络)中的适用性仍 largely 未被探索。本研究深入探讨了通用损失函数下基于Kullback Leibler散度的鲁棒满意模型,给出了解析解释、多种性能保证、高效稳定的数值方法、收敛性分析,以及针对层次化数据结构的扩展方案。通过在三个不同机器学习任务上开展的大量数值实验,我们证明了该模型相较于当前最优基准的更优性能。

英文摘要:

Empirical risk minimization, a cornerstone in machine learning, is often hindered by the Optimizer's Curse stemming from discrepancies between the empirical and true data-generating distributions.To address this challenge, the robust satisficing framework has emerged recently to mitigate ambiguity in the true distribution. Distinguished by its interpretable hyperparameter and enhanced performance guarantees, this approach has attracted increasing attention from academia. However, its applicability in tackling general machine learning problems, notably deep neural networks, remains largely unexplored due to the computational challenges in solving this model efficiently across general loss functions. In this study, we delve into the Kullback Leibler divergence based robust satisficing model under a general loss function, presenting analytical interpretations, diverse performance guarantees, efficient and stable numerical methods, convergence analysis, and an extension tailored for hierarchical data structures. Through extensive numerical experiments across three distinct machine learning tasks, we demonstrate the superior performance of our model compared to state-of-the-art benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑