协变量偏移的福与祸：对抗学习动力学、方向收敛性与均衡态
Blessings and Curses of Covariate Shifts: Adversarial Learning Dynamics, Directional Convergence, and Equilibria
AI总结：
该研究在无穷维框架下刻画对抗协变量偏移的外推区域，揭示其对回归和分类任务均衡学习的相反效应：回归中协变量指数收敛至最优实验设计（福），分类中则次二次收敛至最难设计（祸）。
摘要：
协变量分布偏移与对抗扰动给传统统计学习框架带来了鲁棒性挑战：测试协变量分布的轻微偏移就可能显著影响基于训练分布学习得到的统计模型的性能。当发生外推时，模型性能通常会下降：即协变量偏移到训练分布稀疏的区域，自然地，已学习的模型几乎没有相关信息。出于鲁棒性和正则化的考量，对抗扰动技术被提出作为一种解决方案；然而，对于给定一个已学习模型的情况下，对抗协变量偏移会聚焦于哪个外推区域，仍需开展细致研究。本文精准刻画了外推区域，在无穷维设置下同时考察了回归和分类任务。我们在序贯博弈框架中研究了对抗协变量偏移对后续均衡学习（即贝叶斯最优模型）的影响。我们利用对抗学习博弈的动力学特性，揭示了协变量偏移对均衡学习和实验设计的奇特效应。具体而言，我们建立了两个呈现独特现象的方向收敛结果：（1）回归任务中的“福”：对抗协变量以指数速率偏移到最优实验设计，以加速后续学习；（2）分类任务中的“祸”：对抗协变量以次二次速率偏移到最难的实验设计，困住后续学习进程。
英文摘要：
Covariate distribution shifts and adversarial perturbations present robustness challenges to the conventional statistical learning framework: mild shifts in the test covariate distribution can significantly affect the performance of the statistical model learned based on the training distribution. The model performance typically deteriorates when extrapolation happens: namely, covariates shift to a region where the training distribution is scarce, and naturally, the learned model has little information. For robustness and regularization considerations, adversarial perturbation techniques are proposed as a remedy; however, careful study needs to be carried out about what extrapolation region adversarial covariate shift will focus on, given a learned model. This paper precisely characterizes the extrapolation region, examining both regression and classification in an infinite-dimensional setting. We study the implications of adversarial covariate shifts to subsequent learning of the equilibrium -- the Bayes optimal model -- in a sequential game framework. We exploit the dynamics of the adversarial learning game and reveal the curious effects of the covariate shift to equilibrium learning and experimental design. In particular, we establish two directional convergence results that exhibit distinctive phenomena: (1) a blessing in regression, the adversarial covariate shifts in an exponential rate to an optimal experimental design for rapid subsequent learning; (2) a curse in classification, the adversarial covariate shifts in a subquadratic rate to the hardest experimental design trapping subsequent learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
