Empirical risk minimization (ERM) is known in practice to be non-robust to distributional shift where the training and the test distributions are different. A suite of approaches, such as importance weighting, and variants of distributionally robust optimization (DRO), have been proposed to solve this problem. But a line of recent work has empirically shown that these approaches do not significantly improve over ERM in real applications with distribution shift. The goal of this work is to obtain a comprehensive theoretical understanding of this intriguing phenomenon. We first posit the class of Generalized Reweighting (GRW) algorithms, as a broad category of approaches that iteratively update model parameters based on iterative reweighting of the training samples. We show that when overparameterized models are trained under GRW, the resulting models are close to that obtained by ERM. We also show that adding small regularization which does not greatly affect the empirical training accuracy does not help. Together, our results show that a broad category of what we term GRW approaches are not able to achieve distributionally robust generalization. Our work thus has the following sobering takeaway: to make progress towards distributionally robust generalization, we either have to develop non-GRW approaches, or perhaps devise novel classification/regression loss functions that are adapted to the class of GRW approaches.
Deep classifiers are known to rely on spurious features $\unicode{x2013}$ patterns which are correlated with the target on the training data but not inherently relevant to the learning problem, such as the image backgrounds when classifying the foregrounds. In this paper we evaluate the amount of information about the core (non-spurious) features that can be decoded from the representations learned by standard empirical risk minimization (ERM) and specialized group robustness training. Following recent work on Deep Feature Reweighting (DFR), we evaluate the feature representations by re-traini…
A data set sampled from a certain population is biased if the subgroups of the population are sampled at proportions that are significantly different from their underlying proportions. Training machine learning models on biased data sets requires correction techniques to compensate for the bias. We consider two commonly-used techniques, resampling and reweighting, that rebalance the proportions of the subgroups to maintain the desired objective function. Though statistically equivalent, it has been observed that resampling outperforms reweighting when combined with stochastic gradient algorith…
On the Stability and Convergence of Flow Map Training
Inria, ENS de Lyon, CNRS, Université Claude Bernard Lyon 1（法国国家信息与自动化研究所，里昂高等师范学院，法国国家科学研究中心，克洛德·贝尔纳里昂大学）；Université de Montréal（蒙特利尔大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Imperial College London（伦敦帝国理工学院）；CIFAR AI Chair（加拿大联邦信息研究基金会人工智能主席）
Distributional Biases in Post-Training: A Markovian Analysis of Reasoning Trajectories
City University of Hong Kong（香港城市大学）；Center for Advanced Intelligence Project, RIKEN（RIKEN高级智能研究中心）；The Institute of Statistical Mathematics（统计数学研究所）；University of Sydney（悉尼大学）；CFAR and IHPC, Agency for Science, Technology and Research (A*STAR)（A*STAR的CFAR和IHPC）；Nanyang Technological University（南洋理工大学）；The University of Tokyo（东京大学）
