基于分位数风险最小化的概率域泛化
Probable Domain Generalization via Quantile Risk Minimization
AI总结:
针对域泛化中平均情况预测鲁棒性不足、最坏情况预测过于保守的问题,提出基于分位数风险最小化的概率框架及经验算法,提供理论保证并在基准数据集上取得SOTA表现。
摘要:
域泛化(DG)旨在利用从多个相关训练分布或域中采集的数据,学习在未见过的测试分布上表现良好的预测器。为实现这一目标,DG通常被建模为可能域集合上的平均情况或最坏情况问题。然而,平均表现良好的预测器缺乏鲁棒性,而最坏情况下表现良好的预测器往往过于保守。为解决该问题,我们提出了一种新的DG概率框架,其目标是学习以高概率表现良好的预测器。我们的核心思想是:训练期间观察到的分布偏移应能为我们提供测试时可能出现的偏移的信息,我们通过将训练域和测试域明确关联为来自同一底层元分布的采样来实现这一点。为实现概率DG,我们提出了一种名为分位数风险最小化(QRM)的新优化问题。通过最小化预测器在各域上风险分布的α分位数,QRM旨在寻找以概率α表现良好的预测器。为在实际中求解QRM,我们提出了经验QRM(EQRM)算法,并提供了:(i)EQRM的泛化界;以及(ii)当α趋近于1时EQRM恢复因果预测器的条件。在实验中,我们为DG引入了一种更全面的以分位数为核心的评估协议,并证明EQRM在WILDS和DomainBed的数据集上优于当前最优基线。
英文摘要:
Domain generalization (DG) seeks predictors which perform well on unseen test distributions by leveraging data drawn from multiple related training distributions or domains. To achieve this, DG is commonly formulated as an average- or worst-case problem over the set of possible domains. However, predictors that perform well on average lack robustness while predictors that perform well in the worst case tend to be overly-conservative. To address this, we propose a new probabilistic framework for DG where the goal is to learn predictors that perform well with high probability. Our key idea is that distribution shifts seen during training should inform us of probable shifts at test time, which we realize by explicitly relating training and test domains as draws from the same underlying meta-distribution. To achieve probable DG, we propose a new optimization problem called Quantile Risk Minimization (QRM). By minimizing the $α$-quantile of predictor's risk distribution over domains, QRM seeks predictors that perform well with probability $α$. To solve QRM in practice, we propose the Empirical QRM (EQRM) algorithm and provide: (i) a generalization bound for EQRM; and (ii) the conditions under which EQRM recovers the causal predictor as $α\to 1$. In our experiments, we introduce a more holistic quantile-focused evaluation protocol for DG and demonstrate that EQRM outperforms state-of-the-art baselines on datasets from WILDS and DomainBed.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。