Generalization in generative modeling is defined as the ability to learn an underlying distribution from a finite dataset and produce novel samples, with evaluation largely driven by held-out performance and perceived sample quality. In practice, training is often stopped at the minimum of the test loss, taken as an operational indicator of generalization. We challenge this viewpoint by identifying a phase of biased generalization during training, in which the model continues to decrease the test loss while favoring samples with anomalously high proximity to training data. By training the same network on two disjoint datasets and comparing the mutual distances of generated samples and their similarity to training data, we introduce a quantitative measure of bias and demonstrate its presence on real images. We then study the mechanism of bias, using a controlled hierarchical data model where access to exact scores and ground-truth statistics allows us to precisely characterize its onset. We attribute this phenomenon to the sequential nature of feature learning in deep networks, where coarse structure is learned early in a data-independent manner, while finer features are resolved later in a way that increasingly depends on individual training samples. Our results show that early stopping at the test loss minimum, while optimal under standard generalization criteria, may be insufficient for privacy-critical applications.
Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis
Department of Computer Science, University of Toronto（计算机科学系，多伦多大学）；Department of Statistical Sciences, University of Toronto（统计科学系，多伦多大学）；Data Science Institute（数据科学研究所）；Vector Institute（向量研究所）；Robotics Institute（机器人研究所）
FADE: Towards Fairness-aware Generation for Domain Generalization via Classifier-Guided Score-based Diffusion Models
School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）；School of Informatics, Xiamen University（厦门大学信息学院）；Department of Computer Science, Baylor University（贝勒大学计算机科学系）；Departments of Biostatistics and Epidemiology, Harvard University（哈佛大学生物统计学与流行病学系）
Loop-Diffusion: an equivariant diffusion model for designing and scoring protein loops
University of Washington（华盛顿大学）；Paul G. Allen School of Computer Science and Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Department of Applied Mathematics, University of Washington（华盛顿大学应用数学系）；Fred Hutch Cancer Research Center（弗雷德·哈钦森癌症研究中心）
Sluggish and Chemically-Biased Interstitial Diffusion in Concentrated Solid Solution Alloys: Mechanisms and Methods
City University of Hong Kong（香港城市大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）；Hong Kong Institute for Advanced Study, City University of Hong Kong（香港城市大学香港高等研究院）
Towards Combating Frequency Simplicity-biased Learning for Domain Generalization
Computer Vision Institute（计算机视觉研究所）；School of Computer Science & Software Engineering（计算机科学与软件工程学院）；Shenzhen University（深圳大学）；Shenzhen Institute of Artificial Intelligence Robotics for Society（深圳人工智能与机器人社会研究院）；Guangdong Provincial Key Laboratory of Intelligent Information Processing（广东省智能信息处理重点实验室）；University of Exeter（埃克塞特大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Expressivity and Generalization: Fragment-Biases for Molecular GNNs
School of Computation, Information & Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Munich Data Science Institute（慕尼黑数据科学研究所）；Helmholtz Center for Computational Health（亥姆霍兹计算健康中心）
Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RL
Institute for Interdisciplinary Information Sciences（交叉信息学院）；Tsinghua University（清华大学）；Department of Electronic and Computer Engineering（电子与计算机工程系）；Hong Kong University of Science and Technology（香港科学与技术大学）
