Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systematic benchmark of Jiang et al. (2020) evaluated many generalization measures, but it focused on independent and identically distributed (IID) settings. We revisit this problem for image classifiers evaluated under controlled corruptions and perturbations. Our study uses CIFAR-10-C/P, where the label space and task remain fixed while the input images are degraded or perturbed. This setting also allows us to revisit the robustness concerns raised by Dziugaite et al. (2020), who showed that the apparent reliability of generalization measures can depend strongly on experimental conditions. Our experiments show that the usefulness of generalization measures is strongly regime-dependent. In our exploratory decision analysis across three CNN-style architectures, sharpness- and input-gradient-based measures are among the leading individual signals, whereas family results are close and architecture dependent. Optimization-based measures, Information Criteria, and Sharpness-based measures provide additional regime-dependent signals in correlation or local-reliability analyses. Together, these findings suggest that model selection should not rely only on measures favored by IID evaluation. Instead, within the evaluated CIFAR-10-C/P setting and architectures, generalization measures should be treated as regime-dependent ranking signals whose utility must be evaluated for the intended corruption or perturbation setting.
Multi-CATE: Multi-Accurate Conditional Average Treatment Effect Estimation Robust to Unknown Covariate Shifts
Ludwig-Maximilians-University of Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Cornell University（康奈尔大学）；University of Southern California（南加州大学）
GDDA: Semantic OOD Detection on Graphs under Covariate Shift via Score-Based Diffusion Models
School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）；Baylor University（贝勒大学）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
Separating Covariate Shift from Mechanism Change with Two Discriminators: CJSD, a Conditional Discrepancy with an Exact Covariate-Concept Decomposition
Center for Management of Information Technologies, Kagoshima University（鹿儿岛大学信息技术管理中心）
Learning covariate importance for matching in policy-relevant observational research
Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA（生物统计学、流行病学与信息学系，宾夕法尼亚大学佩雷尔曼医学学院，费城，PA）；School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China（数据科学学院，香港中文大学（深圳）（CUHK-Shenzhen），中国）
TILT: Target-induced loss tilting under covariate shift
Lab for Information and Decision Systems（信息与决策系统实验室）；Statistics and Data Science Center（统计与数据科学中心）；EECS, Massachusetts Institute of Technology（麻省理工学院电子工程与计算机科学系）；Mathematics and EECS, Massachusetts Institute of Technology（数学与电子工程与计算机科学系, 麻省理工学院）
One-Shot Price Forecasting with Covariate-Guided Experts under Privacy Constraints
Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Ltd.（有限公司）；Guangdong Power Grid Co.（广东电网公司）；University of Canterbury（坎特伯雷大学）；Department of Electrical（电气工程系）；Department of Electrical Engineering（电气工程系）
