支持灵活样本查询的组分布鲁棒优化
Group Distributionally Robust Optimization with Flexible Sample Queries
AI总结：
针对现有组分布鲁棒优化算法无法适配动态样本量的问题，将其建模为双人博弈，提出新型PLA算法与支持可变样本量的GDRO算法，推导高概率误差界并通过实验验证有效性。
摘要：
组分布鲁棒优化（Group Distributionally Robust Optimization, GDRO）旨在构建能同时在$m$个分布上表现良好的模型。现有GDRO算法每次迭代仅能处理固定数量的样本，要么是1个要么是$m$个，因此无法支持样本量动态变化的场景。为解决这一局限，我们研究了支持灵活样本查询的GDRO问题，并将其建模为一个双人博弈：其中一个参与方求解在线凸优化问题，另一个参与方求解有限建议预测（Prediction with Limited Advice, PLA）问题。在该博弈框架下，我们提出了一种新的PLA算法，针对样本量为1和不为1的情况分别构建合适的损失估计器，并采用跟随正则化领导者（follow-the-regularized-leader）方法更新决策。随后，我们首次建立了非遗忘（non-oblivious）PLA的高概率悔界。基于上述方法，我们开发了一种每轮可使用任意可变样本量的GDRO算法，其高概率优化误差界为$Oleft( rac{1}{t}sqrt{sum_{j=1}^t rac{m}{r_j}log m} ight)$，其中$r_t$表示第$t$轮的样本量。该结果表明优化误差随样本数量增加而降低，且对于任意固定样本量$rin[m]$，样本复杂度一致为$O(mlog (m)/ε^2)$，与现有$r=1$或$m$情形下的界相符。我们在合成二分类数据集和真实世界多分类数据集上验证了所提方法的有效性。
英文摘要：
Group distributionally robust optimization (GDRO) aims to learn models that perform well across $m$ distributions simultaneously. Existing methods either (i) query $m$ samples per round, which imposes strict implementation requirements, or (ii) query only a single sample per round, which requires many rounds to converge and therefore leads to long runtime. To avoid these limitations, we introduce a novel setting called \emph{flexible sample queries} (FSQ), which allows the number of samples that can be queried to vary across rounds. Under the FSQ setting, we cast GDRO as a two-player game: one performs non-oblivious online convex optimization, while the other tackles a non-oblivious prediction with limited advice (PLA) problem. In this game, the first player updates its decision using follow-the-regularized-leader with stochastic gradients. For the second player, we develop a new PLA method equipped with an adaptive exploration strategy that can leverage an arbitrary per-round sample size. We then establish the \emph{first} high-probability regret bound for non-oblivious PLA. By integrating these components, we develop an anytime GDRO algorithm that supports FSQ, and derive a high-probability optimization error bound of $O\left(\frac{1}{t}\sqrt{\sum_{j=1}^t \frac{m}{r_j}\log m}\right)$, where $r_j$ is the sample size at round $j$. This result demonstrates that the optimization error decreases as the per-round sample sizes increase, and implies the same near-optimal sample complexity of $O(m\log (m)/ε^2)$ for any fixed sample size $r\in[m]$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
