Recently, surrogate models based on deep learning have attracted much attention for engineering analysis and optimization. As the construction of data pairs in most engineering problems is time-consuming, data acquisition is becoming the predictive capability bottleneck of most deep surrogate models, which also exists in surrogate for thermal analysis and design. To address this issue, this paper develops a physics-informed convolutional neural network (CNN) for the thermal simulation surrogate. The network can learn a mapping from heat source layout to the steady-state temperature field without labeled data, which equals solving an entire family of partial difference equations (PDEs). To realize the physics-guided training without labeled data, we employ the heat conduction equation and finite difference method to construct the loss function. Since the solution is sensitive to boundary conditions, we properly impose hard constraints by padding in the Dirichlet and Neumann boundary conditions. In addition, the neural network architecture is well-designed to improve the prediction precision of the problem at hand, and pixel-level online hard example mining is introduced to overcome the imbalance of optimization difficulty in the computation domain. The experiments demonstrate that the proposed method can provide comparable predictions with numerical method and data-driven deep learning models. We also conduct various ablation studies to investigate the effectiveness of the network component and training methods proposed in this paper.
Numerical methods such as finite element have been flourishing in the past decades for modeling solid mechanics problems via solving governing partial differential equations (PDEs). A salient aspect that distinguishes these numerical methods is how they approximate the physical fields of interest. Physics-informed deep learning is a novel approach recently developed for modeling PDE solutions and shows promise to solve computational mechanics problems without using any labeled data. The philosophy behind it is to approximate the quantity of interest (e.g., PDE solution variables) by a deep neu…
Physics-Informed Deep Learning and Partial Transfer Learning for Bearing Fault Diagnosis in the Presence of Highly Missing Data
Faculty of Electrical and Computer Engineering, Tarbiat Modares University（塔里克·莫达雷斯大学电气与计算机工程学院）；Faculty of Technology and Engineering, University of Mazandaran（马赞德兰大学技术与工程学院）
Physics-informed self-supervised learning for predictive modeling of coronary artery digital twins
Chair of Mathematical Data Science, EPFL, Lausanne, Switzerland（数学数据科学教授职位，苏黎世联邦理工学院，拉沃斯，瑞士）；LTS4 laboratory, EPFL, Lausanne, Switzerland（LTS4实验室，苏黎世联邦理工学院，拉沃斯，瑞士）；School of AI and Advanced Computing, Xi’an Jiaotong-Liverpool University, China（人工智能与高级计算学院，西安交通大学利物浦大学，中国）；Cardiology Department, Lausanne University Center Hospital, Lausanne, Switzerland（心血管科，拉沃斯大学中心医院，拉沃斯，瑞士）；OLV Hospital, Aalst, Belgium（OLV医院，阿尔斯特，比利时）；AI Center, EPFL, Lausanne, Switzerland（人工智能中心，苏黎世联邦理工学院，拉沃斯，瑞士）
Technical University of Denmark（丹麦技术大学）；Ørsted Wind Power（沃旭风电）
A significant increase in renewable energy production is necessary to achieve the UN's net-zero emission targets for 2050. Using power-electronic controllers, such as Phase Locked Loops (PLLs), to keep grid-tied renewable resources in synchronism with the grid can cause fast transient behavior during grid faults leading to instability. However, assessing all the probable scenarios is impractical, so determining the stability boundary or region of attraction (ROA) is necessary. However, using EMT simulations or Reduced-order models (ROMs) to accurately determine the ROA is computationally expen…
Conservation-informed Graph Learning for Spatiotemporal Dynamics Prediction
Renmin University of China（中国人民大学）；Northeastern University（东北大学）；Huawei Technologies（华为技术有限公司）；HKUST（香港科技大学）；University of Chinese Academy of Sciences（中国科学院大学）
Enhancing Document AI Data Generation Through Graph-Based Synthetic Layouts
Liverpool John Moores University（利物浦约翰摩尔斯大学）；University of Washington（华盛顿大学）；Columbia University（哥伦比亚大学）；New York University（纽约大学）；Birla Institute of Technology（贝拉理工学院）
Recent years have seen flourishing research on both semi-supervised learning and 3D room layout reconstruction. In this work, we explore the intersection of these two fields to advance the research objective of enabling more accurate 3D indoor scene modeling with less labeled data. We propose the first approach to learn representations of room corners and boundaries by using a combination of labeled and unlabeled data for improved layout estimation in a 360-degree panoramic scene. Through extensive comparative experiments, we demonstrate that our approach can advance layout estimation of compl…
Visualizing High-Dimensional Graph Embeddings via Informed Multi-View Projections
Khoury College of Computer Sciences, Northeastern University（东北大学库里计算机科学学院）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）
By integrating domain knowledge with labeled samples, informed machine learning has been emerging to improve the learning performance for a wide range of applications. Nonetheless, rigorous understanding of the role of injected domain knowledge has been under-explored. In this paper, we consider an informed deep neural network (DNN) with over-parameterization and domain knowledge integrated into its training objective function, and study how and why domain knowledge benefits the performance. Concretely, we quantitatively demonstrate the two benefits of domain knowledge in informed learning - r…
National Innovation Institute of Defense Technology, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）
Physics-informed neural networks (PINNs) have been widely used to solve various scientific computing problems. However, large training costs limit PINNs for some real-time applications. Although some works have been proposed to improve the training efficiency of PINNs, few consider the influence of initialization. To this end, we propose a New Reptile initialization based Physics-Informed Neural Network (NRPINN). The original Reptile algorithm is a meta-learning initialization method based on labeled data. PINNs can be trained with less labeled data or even without any labeled data by adding p…
This paper presents MixText, a semi-supervised learning method for text classification, which uses our newly designed data augmentation method called TMix. TMix creates a large amount of augmented training samples by interpolating text in hidden space. Moreover, we leverage recent advances in data augmentation to guess low-entropy labels for unlabeled data, hence making them as easy to use as labeled data.By mixing labeled, unlabeled and augmented data, MixText significantly outperformed current pre-trained and fined-tuned models and other state-of-the-art semi-supervised learning methods on s…
College of Aerospace Science and Engineering National University of Defense Technology（国防科技大学航空航天学院）；Defense Innovation Institute Chinese Academy of Military Science（中国人民解放军军事科学院国防创新研究院）
For the temperature field reconstruction (TFR), a complex image-to-image regression problem, the convolutional neural network (CNN) is a powerful surrogate model due to the convolutional layer's good image feature extraction ability. However, a lot of labeled data is needed to train CNN, and the common CNN can not quantify the aleatoric uncertainty caused by data noise. In actual engineering, the noiseless and labeled training data is hardly obtained for the TFR. To solve these two problems, this paper proposes a deep Monte Carlo quantile regression (Deep MC-QR) method for reconstructing the t…
Unsupervised Physics-Informed Operator Learning through Multi-Stage Curriculum Training
organization= Department of Civil；organization= QC Ware Corp. , city= Paris , country= France；organization= Department of Computer Science, KTH Royal Institute of Technology , city= Stockholm , country= Sweden；organization= IRIF, CNRS \& Université Paris Cité , city= Paris , country= France
In typical machine learning tasks and applications, it is necessary to obtain or create large labeled datasets in order to to achieve high performance. Unfortunately, large labeled datasets are not always available and can be expensive to source, creating a bottleneck towards more widely applicable machine learning. The paradigm of weak supervision offers an alternative that allows for integration of domain-specific knowledge by enforcing constraints that a correct solution to the learning problem will obey over the output space. In this work, we explore the application of this paradigm to 2-D…
Rethinking Physics-Informed Regression Beyond Training Loops and Bespoke Architectures
Department of Electrical \& Electronic Engineering, Imperial College, SW7 2AZ London, UK (e-mail: ).；Department of Aeronautics, Imperial College, \ 7 2AZ London, UK (e-mail: )
PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations
Department of Statistics and Data Science, Yale University（统计与数据科学系，耶鲁大学）；Department of Applied Mathematics and Statistics, Johns Hopkins University（应用数学与统计学系，约翰霍普金斯大学）；Department of Applied Computational Mathematics and Statistics, University of Notre Dame（应用计算数学与统计学系，圣母大学）；Department of Mathematics, University of California Los Angeles（数学系，加州大学洛杉矶分校）；Department of Chemical and Environmental Engineering, Yale University（化学与环境工程系，耶鲁大学）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Livermore Software Technology, an ANSYS company（利弗莫尔软件技术公司，ANSYS 旗下公司）
The recent explosion of machine learning (ML) and artificial intelligence (AI) shows great potential in the breakthrough of metal additive manufacturing (AM) process modeling. However, the success of conventional machine learning tools in data science is primarily attributed to the unprecedented large amount of labeled data-sets (big data), which can be either obtained by experiments or first-principle simulations. Unfortunately, these labeled data-sets are expensive to obtain in AM due to the high expense of the AM experiments and prohibitive computational cost of high-fidelity simulations. W…
PICore: Physics-Informed Unsupervised Coreset Selection for Data Efficient Neural Operator Training
Department of Computer Science（计算机科学系）；University of Maryland, College Park（马里兰大学学院公园分校）；Georgia Institute of Technology（佐治亚理工学院）；Department of Mathematics（数学系）；Center for Scientific Computation and Mathematical Modeling（科学计算与数学建模中心）
Label-Informed Outlier Detection Based on Granule Density
College of Computer Science, Sichuan University（四川大学计算机学院）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机与人工智能学院）；Sichuan National Innovation New Vision UHD Video Technology Co., Ltd.（四川国家创新新视野UHD视频技术有限公司）
Partition of Unity Physics-Informed Neural Networks (POU-PINNs): An Unsupervised Framework for Physics-Informed Domain Decomposition and Mixtures of Experts
The University of Texas at El Paso（德克萨斯大学埃尔帕索分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Clarkson University（克拉克森大学）；Texas A&M University in Kingsville（德克萨斯农工大学金斯维尔分校）
Deep neural networks have recently led to promising results for the task of multiple sound source localization. Yet, they require a lot of training data to cover a variety of acoustic conditions and microphone array layouts. One can leverage acoustic simulators to inexpensively generate labeled training data. However, models trained on synthetic data tend to perform poorly with real-world recordings due to the domain mismatch. Moreover, learning for different microphone array layouts makes the task more complicated due to the infinite number of possible layouts. We propose to use adversarial l…
Knowledge-Informed Machine Learning for Cancer Diagnosis and Prognosis: A review
Georgia Institute of Technology（佐治亚理工学院）；Mayo Clinic Arizona（妙佑医疗国际亚利桑那州院区）；Arizona State University（亚利桑那州立大学）；Columbia University Medical Center（哥伦比亚大学医学中心）
Multi-Joint Physics-Informed Deep Learning Framework for Time-Efficient Inverse Dynamics
School of Electronic and Electrical Engineering, University of Leeds（利兹大学电子与电气工程学院）；Key Laboratory of Mechanism Theory and Equipment Design of the Ministry of Education, School of Mechanical Engineering, Tianjin University（教育部机制理论与装备设计重点实验室，天津大学机械工程学院）；College of Intelligent Systems Science and Engineering, Hubei Minzu University（湖北民族大学智能系统科学与工程学院）
Indian Institute of Technology (IIT) Delhi（印度理工学院德里分校）
Deep neural operators are recognized as an effective tool for learning solution operators of complex partial differential equations (PDEs). As compared to laborious analytical and computational tools, a single neural operator can predict solutions of PDEs for varying initial or boundary conditions and different inputs. A recently proposed Wavelet Neural Operator (WNO) is one such operator that harnesses the advantage of time-frequency localization of wavelets to capture the manifolds in the spatial domain effectively. While WNO has proven to be a promising method for operator learning, the dat…
In this paper, we introduce a simple method that can separate arbitrary musical instruments from an audio mixture. Given an unaligned MIDI transcription for a target instrument from an input mixture, we synthesize new mixtures from the midi transcription that sound similar to the mixture to be separated. This lets us create a labeled training set to train a network on the specific bespoke task. When this model applied to the original mixture, we demonstrate that this method can: 1) successfully separate out the desired instrument with access to only unaligned MIDI, 2) separate arbitrary instru…
We develop an approach that benefits from large simulated datasets and takes full advantage of the limited online data that is most relevant. We propose a variant of Bayesian optimization that alternates between using informed and uninformed kernels. With this Bernoulli Alternation Kernel we ensure that discrepancies between simulation and reality do not hinder adapting robot control policies online. The proposed approach is applied to a challenging real-world problem of task-oriented grasping with novel objects. Our further contribution is a neural network architecture and training pipeline t…
NASA Harvest（美国国家航空航天局收获计划）；McGill University（麦吉尔大学）；Mila – Quebec AI Institute（米拉-魁北克人工智能研究所）；University of Maryland, College Park（马里兰大学帕克分校）
Labeled datasets for agriculture are extremely spatially imbalanced. When developing algorithms for data-sparse regions, a natural approach is to use transfer learning from data-rich regions. While standard transfer learning approaches typically leverage only direct inputs and outputs, geospatial imagery and agricultural data are rich in metadata that can inform transfer learning algorithms, such as the spatial coordinates of data-points or the class of task being learned. We build on previous work exploring the use of meta-learning for agricultural contexts in data-sparse regions and introduc…
Physics-informed generative AI for semiconductor manufacturing: Enforcing hard physical constraints in generative models by construction
School of Electrical and Computer Engineering, University of Oklahoma（俄克拉荷马大学电气与计算机工程学院）；Center for Quantum Research and Technology, University of Oklahoma（俄克拉荷马大学量子研究与技术中心）；Intelligent Neuromorphic and Quantum Understanding for Innovative Research and Engineering (INQUIRE) Laboratory（创新研究与工程智能神经形态与量子理解实验室）；Material Science and Engineering Program, University of Oklahoma, Norman, OK 73019 USA（俄克拉荷马大学材料科学与工程项目，Norman, OK 73019 USA）
A unified physics-informed generative operator framework for general inverse problems
Zhejiang University, Center for Interdisciplinary Applied Mathematics（浙江大学，交叉应用数学中心）；Technical University of Munich, Professorship of Data-driven Materials Modeling, School of Engineering and Design（慕尼黑技术大学，数据驱动材料建模教授职位，工程与设计学院）
College of Aerospace Science and Engineering, National University of Defense Technology（国防科技大学航空航天学院）；Defense Innovation Institute, Chinese Academy of Military Science（中国人民解放军军事科学院国防创新研究院）
Temperature field reconstruction is essential for analyzing satellite heat reliability. As a representative machine learning model, the deep convolutional neural network (DCNN) is a powerful tool for reconstructing the satellite temperature field. However, DCNN needs a lot of labeled data to learn its parameters, which is contrary to the fact that actual satellite engineering can only acquire noisy unlabeled data. To solve the above problem, this paper proposes an unsupervised method, i.e., the physics-informed deep Monte Carlo quantile regression method, for reconstructing temperature field a…
Christian Doppler Laboratory for Magnet design through physics informed machine learning（通过物理感知机器学习进行磁体设计的克里斯蒂安·多普勒实验室）；Danube University Krems（克雷姆斯多瑙大学）；Department of Mathematics, University of Vienna（维也纳大学数学系）；Wolfgang Pauli Institute（沃尔夫冈·泡利研究所）；Research Platform MMM Mathematics-Magnetism-Materials（MMM数学-磁学-材料研究平台）
We introduce conditional PINNs (physics informed neural networks) for estimating the solution of classes of eigenvalue problems. The concept of PINNs is expanded to learn not only the solution of one particular differential equation but the solutions to a class of problems. We demonstrate this idea by estimating the coercive field of permanent magnets which depends on the width and strength of local defects. When the neural network incorporates the physics of magnetization reversal, training can be achieved in an unsupervised way. There is no need to generate labeled training data. The present…
Solving Oscillator Ordinary Differential Equations in the Time Domain with High Performance via Soft-constrained Physics-informed Neural Network with Small Data
College of Information Technology, Shanghai Jian Qiao University（上海建桥学院信息技术学院）；School of Urban Railway Transportation, Shanghai University of Engineering Science（上海工程技术大学城市轨道交通学院）
Model-based and learning-based methods are two major types of methodologies to model car following behaviors. Model-based methods describe the car-following behaviors with explicit mathematical equations, while learning-based methods focus on getting a mapping between inputs and outputs. Both types of methods have advantages and weaknesses. Meanwhile, most car-following models are generative and only consider the inputs of the speed, position, and acceleration of the last time step. To address these issues, this study proposes a novel framework called IDM-Follower that can generate a sequence …
