We present a non-convex optimization algorithm metaheuristic, based on the training of a deep generative network, which enables effective searching within continuous, ultra-high dimensional landscapes. During network training, populations of sampled local gradients are utilized within a customized loss function to evolve the network output distribution function towards one peak at high-performing optima. The deep network architecture is tailored to support progressive growth over the course of training, which allows the algorithm to manage the curse of dimensionality characteristic of high-dimensional landscapes. We apply our concept to a range of standard optimization problems with dimensions as high as one thousand and show that our method performs better with fewer function evaluations compared to state-of-the-art algorithm benchmarks. We also discuss the role of deep network over-parameterization, loss function engineering, and proper network architecture selection in optimization, and why the required batch size of sampled local gradients is independent of problem dimension. These concepts form the foundation for a new class of algorithms that utilize customizable and expressive deep generative networks to solve non-convex optimization problems.
Predicting Landscapes from Environmental Conditions Using Generative Networks
Institute of Data Science（数据科学研究所）；German Aerospace Center (DLR)（德国航空航天中心）；Max-Planck-Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）；Friedrich-Schiller-Universität（弗里德里希-席勒大学）
Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes
Department of Computer Science, Yonsei University, Seoul, Korea（计算机科学系，延世大学，首尔，韩国）；Department of Artificial Intelligence, Yonsei University, Seoul, Korea（人工智能系，延世大学，首尔，韩国）
Energy-Guided Generative Modeling for Low-Energy Molecular Structure Discovery
School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
SPIBER: Reconstructing Free Energy Landscapes from Short, Unconverged Trajectories with Generative Flow Networks
University of Maryland（马里兰大学）；Institute for Physical Science and Technology（物理科学与技术研究所）；University of Maryland Institute for Health Computing（马里兰大学健康计算研究所）
Exploring the Energy Landscape of RBMs: Reciprocal Space Insights into Bosons, Hierarchical Learning and Symmetry Breaking
TRIUMF；Perimeter Institute for Theoretical Physics（佩里米特理论物理研究所）；University of Virginia（弗吉尼亚大学）；Computer Science and Biocomplexity Institute（计算机科学与生物复杂性研究所）；Department of Physics and Astronomy（物理与天文学系）；University of Waterloo（滑铁卢大学）
Hybrid Adversarial Spectral Loss Conditional Generative Adversarial Networks for Signal Data Augmentation in Ultra-precision Machining Surface Roughness Prediction
X2CT-FLOW: Maximum a posteriori reconstruction using a progressive flow-based deep generative model for ultra sparse-view computed tomography in ultra low-dose protocols
The University of Tokyo Hospital（东京大学医学部附属医院）；Center for Frontier Medical Engineering, Chiba University（千叶大学前沿医疗工程中心）；Division of Radiology and Biomedical Engineering, Graduate School of Medicine, The University of Tokyo（东京大学医学研究生院放射科与生物医学工程学系）
Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks
Centre for Medical Image Computing, University College London（伦敦大学学院医学图像计算中心）；Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Department of Diagnostic Imaging, Royal Free London NHS Foundation Trust（伦敦皇家自由NHS基金会信托诊断影像科）
Semantic Encoder Guided Generative Adversarial Face Ultra-Resolution Network
Lakehead University（湖首大学）；Western University（韦仕敦大学）；Okanagan College（奥肯那根学院）；Hunan Normal University（湖南师范大学）；Tianjin University（天津大学）；The University of British Columbia Okanagan（不列颠哥伦比亚大学奥肯那根分校）
GMem: A Modular Approach for Ultra-Efficient Generative Models
Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心）；School of Engineering, Westlake University（西湖大学工程学院）；Zhejiang University（浙江大学）
