In recent years, great success has been witnessed in building problem-specific deep networks from unrolling iterative algorithms, for solving inverse problems and beyond. Unrolling is believed to incorporate the model-based prior with the learning capacity of deep learning. This paper revisits the role of unrolling as a design approach for deep networks: to what extent its resulting special architecture is superior, and can we find better? Using LISTA for sparse recovery as a representative example, we conduct the first thorough design space study for the unrolled models. Among all possible variations, we focus on extensively varying the connectivity patterns and neuron types, leading to a gigantic design space arising from LISTA. To efficiently explore this space and identify top performers, we leverage the emerging tool of neural architecture search (NAS). We carefully examine the searched top architectures in a number of settings, and are able to discover networks that are consistently better than LISTA. We further present more visualization and analysis to "open the black box", and find that the searched top architectures demonstrate highly consistent and potentially transferable patterns. We hope our study to spark more reflections and explorations on how to better mingle model-based optimization prior and data-driven learning.
ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods
Institute of AI for Health, Helmholtz Munich（亥姆霍兹慕尼黑人工智能健康研究所）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术学院）；Helmholtz AI（亥姆霍兹人工智能研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Faculty of Mathematics, Informatics and Mechanics, University of Warsaw（华沙大学数学、信息学与力学学院）
