arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.20451stat.MLcs.LGmath.OC
收藏

Robust Satisficing的统计性质

Statistical Properties of Robust Satisficing

School of Mathematical Sciences, Peking University(北京大学数学科学学院);Laboratory for Information and Decision Systems, Massachusetts Institute of Technology(麻省理工学院信息与决策系统实验室);Department of Industrial Engineering and Decision Analytics, The Hong Kong University of Science and Technology(香港科技大学工业工程与决策分析系);HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute(香港科技大学深港协同创新研究院)

Zhiyi Li, Yunbei Xu, Ruohan Zhan

AI总结:

本文填补了Robust Satisficing模型统计理论的空白,推导了最优损失双侧置信区间与有限样本泛化误差界限,并证明其在小样本和分布偏移下优于基线且对超参数不敏感。

摘要:

Robust Satisficing (RS) 模型是一种新兴的鲁棒优化方法,在各种应用中提供了简化的流程和鲁棒的泛化能力。然而,文献中尚未探讨RS的统计理论。本文通过全面分析RS模型的理论性质填补了这一空白。值得注意的是,与开创性的Distributionally Robust Optimization (DRO) 相比,RS结构为推导统计保证提供了更直接的路径,从而产生了更丰富的结果。具体而言,我们为最优损失建立了双侧置信区间,而无需显式求解极小极大优化问题。我们进一步为RS优化器提供了有限样本泛化误差界限。重要的是,我们的结果扩展到涉及分布偏移的场景,即采样分布和目标分布之间存在差异的情况。我们的数值实验表明,在小样本机制和分布偏移下,RS模型始终优于基线经验风险最小化。此外,与DRO模型相比,RS模型对超参数调优表现出更低的敏感性,突显了其在鲁棒性考虑方面的实用性。

英文摘要:

The Robust Satisficing (RS) model is an emerging approach to robust optimization, offering streamlined procedures and robust generalization across various applications. However, the statistical theory of RS remains unexplored in the literature. This paper fills in the gap by comprehensively analyzing the theoretical properties of the RS model. Notably, the RS structure offers a more straightforward path to deriving statistical guarantees compared to the seminal Distributionally Robust Optimization (DRO), resulting in a richer set of results. In particular, we establish two-sided confidence intervals for the optimal loss without the need to solve a minimax optimization problem explicitly. We further provide finite-sample generalization error bounds for the RS optimizer. Importantly, our results extend to scenarios involving distribution shifts, where discrepancies exist between the sampling and target distributions. Our numerical experiments show that the RS model consistently outperforms the baseline empirical risk minimization in small-sample regimes and under distribution shifts. Furthermore, compared to the DRO model, the RS model exhibits lower sensitivity to hyperparameter tuning, highlighting its practicability for robustness considerations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑