Deep learning (DL) is revolutionizing the scientific computing community. To reduce the data gap, active learning has been identified as a promising solution for DL in the scientific computing community. However, the deep active learning (DAL) literature is dominated by image classification problems and pool-based methods. Here we investigate the robustness of pool-based DAL methods for scientific computing problems (dominated by regression) where DNNs are increasingly used. We show that modern pool-based DAL methods all share an untunable hyperparameter, termed the pool ratio, denoted $γ$, which is often assumed to be known apriori in the literature. We evaluate the performance of five state-of-the-art DAL methods on six benchmark problems if we assume $γ$ is \textit{not} known - a more realistic assumption for scientific computing problems. Our results indicate that this reduces the performance of modern DAL methods and that they sometimes can even perform worse than random sampling, creating significant uncertainty when used in real-world settings. To overcome this limitation we propose, to our knowledge, the first query synthesis DAL method for regression, termed NA-QBC. NA-QBC removes the sensitive $γ$ hyperparameter and we find that, on average, it outperforms the other DAL methods on our benchmark problems. Crucially, NA-QBC always outperforms random sampling, providing more robust performance benefits.
School of Computer Science & Engineering（计算机科学与工程学院）；South China University of Technology（华南理工大学）
Active learning (AL) is a machine learning algorithm that can achieve greater accuracy with fewer labeled training instances, for having the ability to ask oracles to label the most valuable unlabeled data chosen iteratively and heuristically by query strategies. Scientific experiments nowadays, though becoming increasingly automated, are still suffering from human involvement in the designing process and the exhaustive search in the experimental space. This article performs a retrospective study on a drug response dataset using the proposed AL scheme comprised of the matrix factorization meth…
Deep-learning-based models are increasingly used to emulate scientific simulations to accelerate scientific research. However, accurate, supervised deep learning models require huge amount of labelled data, and that often becomes the bottleneck in employing neural networks. In this work, we leveraged an active learning approach called core-set selection to actively select data, per a pre-defined budget, to be labelled for training. To further improve the model performance and reduce the training costs, we also warm started the training using a shrink-and-perturb trick. We tested on two case st…
Argonne National Laboratory（阿贡国家实验室）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Washington（华盛顿大学）；Stanford University（斯坦福大学）；GrammaTech, Inc.（GrammaTech公司）；University of Wisconsin - Madison（威斯康星大学麦迪逊分校）；Microsoft Corporation（微软公司）；NVIDIA Corporation（英伟达公司）；Brookhaven National Laboratory（布鲁克海文国家实验室）；Carnegie Mellon University（卡内基梅隆大学）；University of Pennsylvania（宾夕法尼亚大学）；Kestrel Institute（红隼研究所）；Massachusetts Institute of Technology（麻省理工学院）；Intel Corporation（英特尔公司）；Texas A&M University（德州农工大学）；University of Arizona（亚利桑那大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Oak Ridge National Laboratory（橡树岭国家实验室）；Center for Computing Sciences（计算科学中心）；University of Oregon（俄勒冈大学）；Stony Brook University（石溪大学）；Boise State Univeristy（博伊西州立大学）；Facebook（脸书）；University of Michigan（密歇根大学）；Grinnell College（格林内尔学院）；University of Utah（犹他大学）；University of California, Berkeley（加州大学伯克利分校）；Amazon（亚马逊）
Program synthesis is an active research field in academia, national labs, and industry. Yet, work directly applicable to scientific computing, while having some impressive successes, has been limited. This report reviews the relevant areas of program synthesis work for scientific computing, discusses successes to date, and outlines opportunities for future work. This report is the result of the Workshop on Program Synthesis for Scientific Computing was held virtually on August 4-5 2020 (https://prog-synth-science.github.io/2020/).
IBM Research（IBM研究院）；University of Minnesota（明尼苏达大学）
The general method of graph coarsening or graph reduction has been a remarkably useful and ubiquitous tool in scientific computing and it is now just starting to have a similar impact in machine learning. The goal of this paper is to take a broad look into coarsening techniques that have been successfully deployed in scientific computing and see how similar principles are finding their way in more recent applications related to machine learning. In scientific computing, coarsening plays a central role in algebraic multigrid methods as well as the related class of multilevel incomplete LU facto…
Julia Computing（Julia Computing公司）；MIT（麻省理工学院）；UMB（马萨诸塞大学波士顿分校）；Invenia Labs（Invenia实验室）
Scientific computing is increasingly incorporating the advancements in machine learning and the ability to work with large amounts of data. At the same time, machine learning models are becoming increasingly sophisticated and exhibit many features often seen in scientific computing, stressing the capabilities of machine learning frameworks. Just as the disciplines of scientific computing and machine learning have shared common underlying infrastructure in the form of numerical linear algebra, we now have the opportunity to further share new computational infrastructure, and thus ideas, in the …
Texas State University（德克萨斯州立大学）；University of Minho（米尼奥大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Texas A&M University（德克萨斯农工大学）
Recent breakthroughs in computing power have made it feasible to use machine learning and deep learning to advance scientific computing in many fields, including fluid mechanics, solid mechanics, materials science, etc. Neural networks, in particular, play a central role in this hybridization. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data is sparse, which is the case in many scientific and engineering domains. Nonetheless, neural networks provide a solid foundation to respect physics-driven or knowledge-based constraints d…
This paper proposes CANC, a Co-teaching Active Noise Cancellation method, applied in spatial computing to address deep learning trained with extreme noisy labels. Deep learning algorithms have been successful in spatial computing for land or building footprint recognition. However a lot of noise exists in ground truth labels due to how labels are collected in spatial computing and satellite imagery. Existing methods to deal with extreme label noise conduct clean sample selection and do not utilize the remaining samples. Such techniques can be wasteful due to the cost of data retrieval. Our pro…
University of Utah（犹他大学）；Tsinghua University（清华大学）；Purdue University（普渡大学）
Diffractive optical neural networks (DONNs) have been emerging as a high-throughput and energy-efficient hardware platform to perform all-optical machine learning (ML) in machine vision systems. However, the current demonstrated applications of DONNs are largely straightforward image classification tasks, which undermines the prospect of developing and utilizing such hardware for other ML applications. Here, we numerically and experimentally demonstrate the deployment of an all-optical reconfigurable DONNs system for scientific computing, including guiding two-dimensional quantum material synt…
Learning task-specific predictive models for scientific computing
Department of Mathematics, National University of Singapore（新加坡国立大学数学系）；Department of Mathematics & Institute for Functional Intelligent Materials, National University of Singapore（新加坡国立大学数学系与功能性智能材料研究所）
Planetary exploration missions with Mars rovers are complicated, which generally require elaborated task planning by human experts, from the path to take to the images to capture. NASA has been using this process to acquire over 22 million images from the planet Mars. In order to improve the degree of automation and thus efficiency in this process, we propose a system for planetary rovers to actively search for prominence of prespecified scientific features in captured images. Scientists can prespecify such search tasks in natural language and upload them to a rover, on which the deployed syst…
Physics-informed learning has shown to have a better generalization than learning without physical priors. However, training physics-informed deep neural networks requires some aspect of physical simulations to be written in a differentiable manner. Unfortunately, some operations and functionals commonly used in physical simulations are scattered, hard to integrate, and lack higher order derivatives which are needed in physical simulations. In this work, we present $ξ$-torch, a library of differentiable functionals for scientific simulations. Example functionals are a root finder and an initia…
AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；NSF National Optical-Infrared Astronomy Research Laboratory（国家光学红外天文研究实验室）；University of Virginia（弗吉尼亚大学）；SLAC National Accelerator Laboratory（斯坦福直线加速器实验室）；Texas Advanced Computing Center（德克萨斯高级计算中心）
Scientific computing applications have benefited greatly from high performance computing infrastructure such as supercomputers. However, we are seeing a paradigm shift in the computational structure, design, and requirements of these applications. Increasingly, data-driven and machine learning approaches are being used to support, speed-up, and enhance scientific computing applications, especially molecular dynamics simulations. Concurrently, cloud computing platforms are increasingly appealing for scientific computing, providing "infinite" computing powers, easier programming and deployment m…
Information Science & Technology Institute (ISTI)（信息科学与技术研究所（ISTI））；Los Alamos National Laboratory (LANL)（洛斯阿拉莫斯国家实验室（LANL））；Amazon Web Services (AWS)（亚马逊网络服务（AWS））；Amazon Web Services（亚马逊网络服务）
This report describes eighteen projects that explored how commercial cloud computing services can be utilized for scientific computation at national laboratories. These demonstrations ranged from deploying proprietary software in a cloud environment to leveraging established cloud-based analytics workflows for processing scientific datasets. By and large, the projects were successful and collectively they suggest that cloud computing can be a valuable computational resource for scientific computation at national laboratories.
Integration of Scanning Probe Microscope with High-Performance Computing: fixed-policy and reward-driven workflows implementation
University of Tennessee（田纳西大学）；Oxford Instruments Asylum Research（牛津仪器庇护研究公司）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；National Cheng Kung University（国立成功大学）；the Pennsylvania State University（宾夕法尼亚州立大学）
