Many success stories involving deep neural networks are instances of supervised learning, where available labels power gradient-based learning methods. Creating such labels, however, can be expensive and thus there is increasing interest in weak labels which only provide coarse information, with uncertainty regarding time, location or value. Using such labels often leads to considerable challenges for the learning process. Current methods for weak-label training often employ standard supervised approaches that additionally reassign or prune labels during the learning process. The information gain, however, is often limited as only the importance of labels where the network already yields reasonable results is boosted. We propose treating weak-label training as an unsupervised problem and use the labels to guide the representation learning to induce structure. To this end, we propose two autoencoder extensions: class activity penalties and structured dropout. We demonstrate the capabilities of our approach in the context of score-informed source separation of music.
Physics-Informed Neural Networks for Methane Sorption: Cross-Gas Transfer Learning, Ensemble Collapse Under Physics Constraints, and Monte Carlo Dropout Uncertainty Quantification
Environmental Geosciences, Department of Geosciences, University of Oslo（奥斯陆大学环境地质学系）；National Key Laboratory of Petroleum Resources and Engineering, China University of Petroleum (Beijing)（中国石油大学（北京）石油资源与工程国家级重点实验室）；College of Geosciences, China University of Petroleum (Beijing)（中国石油大学（北京）地质学院）；State Key Laboratory of Coal Mine Disaster Prevention and Control, China University of Mining and Technology（中国矿业大学煤矿灾害防治控制国家重点实验室）；The Lyell Centre, Heriot-Watt University（赫瑞斯泰大学利尔中心）；Subsurface Energy Transition and Innovation Centre, Heriot-Watt University（赫瑞斯泰大学地下能源转型与创新中心）
Source separation problems are ubiquitous in the physical sciences; any situation where signals are superimposed calls for source separation to estimate the original signals. In this tutorial I will discuss the Bayesian approach to the source separation problem. This approach has a specific advantage in that it requires the designer to explicitly describe the signal model in addition to any other information or assumptions that go into the problem description. This leads naturally to the idea of informed source separation, where the algorithm design incorporates relevant information about the …
In this paper, we introduce a simple method that can separate arbitrary musical instruments from an audio mixture. Given an unaligned MIDI transcription for a target instrument from an input mixture, we synthesize new mixtures from the midi transcription that sound similar to the mixture to be separated. This lets us create a labeled training set to train a network on the specific bespoke task. When this model applied to the original mixture, we demonstrate that this method can: 1) successfully separate out the desired instrument with access to only unaligned MIDI, 2) separate arbitrary instru…
Testing Spintronics Implemented Monte Carlo Dropout-Based Bayesian Neural Networks
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Univ. Grenoble Alpes, CEA, CNRS, Grenoble INP, and IRIG-Spintec（格勒诺布尔阿尔卑斯大学、法国原子能委员会、法国国家科学研究中心、格勒诺布尔国立理工学院及IRIG-Spintec）；RevoAI GmbH（RevoAI有限责任公司）
Brown University（布朗大学）；Pacific Northwest National Laboratory（西北太平洋国家实验室）
We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framewo…
Sorbonne Université（索邦大学）；Grenoble-INP - Institut Polytechnique de Grenoble（格勒诺布尔理工学院）；Télécom Paris - Institut Polytechnique de Paris（巴黎电信学院 - 巴黎理工学院）；Spotify；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；I3S（I3S实验室）；ISIR（智能系统与机器人研究所）；Universitat Pompeu Fabra（庞培法布拉大学）；MTG（音乐技术组）；Centre Inria Nancy - Grand Est（法国国家信息与自动化研究院南锡-大东部研究中心）
This dissertation proposes the study of multimodal learning in the context of musical signals. Throughout, we focus on the interaction between audio signals and text information. Among the many text sources related to music that can be used (e.g. reviews, metadata, or social network feedback), we concentrate on lyrics. The singing voice directly connects the audio signal and the text information in a unique way, combining melody and lyrics where a linguistic dimension complements the abstraction of musical instruments. Our study focuses on the audio and lyrics interaction for targeting source …
STMS UMR9912, Ircam/CNRS/SU（STMS UMR9912，Ircam/CNRS/SU）；LTCI, Institut Polytechnique de Paris（巴黎理工学院 LTCI）
Informed source separation has recently gained renewed interest with the introduction of neural networks and the availability of large multitrack datasets containing both the mixture and the separated sources. These approaches use prior information about the target source to improve separation. Historically, Music Information Retrieval researchers have focused primarily on score-informed source separation, but more recent approaches explore lyrics-informed source separation. However, because of the lack of multitrack datasets with time-aligned lyrics, models use weak conditioning with non-alig…
Solving Euler equations with Multiple Discontinuities via Separation-Transfer Physics-Informed Neural Networks
Graduate School, China Academy of Engineering Physics（中国工程物理研究院研究生院）；China Academy of Engineering Physics（中国工程物理研究院）；Zhejiang Institute of Modern Physics（浙江现代物理研究所）；School of Physics, Zhejiang University（浙江大学物理学院）；Beijing Computational Science Research Center（北京计算科学研究院）
Peking University Shenzhen Graduate School（北京大学深圳研究生院）
Target speech separation refers to extracting the target speaker's speech from mixed signals. Despite the recent advances in deep learning based close-talk speech separation, the applications to real-world are still an open issue. Two main challenges are the complex acoustic environment and the real-time processing requirement. To address these challenges, we propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture in reverberant environments, assisted with directional information of the speaker(s). Firstly, against variations bro…
Uncertainty Quantification In Surface Landmines and UXO Classification Using MC Dropout
Rochester Institute of Technology, Chester F. Carlson Center for Imaging Science（罗切斯特理工学院，切斯特·F·卡森成像科学中心）；Jadavpur University, Department of Computer Science（贾梵布尔大学，计算机科学系）
Pompeu Fabra University（庞培法布拉大学）；University of Waterloo（滑铁卢大学）；Joint Research Centre (EC)（（欧盟委员会）联合研究中心）
Can we perform an end-to-end music source separation with a variable number of sources using a deep learning model? We present an extension of the Wave-U-Net model which allows end-to-end monaural source separation with a non-fixed number of sources. Furthermore, we propose multiplicative conditioning with instrument labels at the bottleneck of the Wave-U-Net and show its effect on the separation results. This approach leads to other types of conditioning such as audio-visual source separation and score-informed source separation.
Biologically Informed Deep Neural Networks for Multi-Omic Integration, Pathway Activity Inference and Risk Stratification in Cancer
King’s College London（伦敦国王学院）；Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（科技研究局信息与通信研究所）；Shenzhen University（深圳大学）；University of Manchester（曼彻斯特大学）
Estimating Dense-Packed Zone Height in Liquid-Liquid Separation: A Physics-Informed Neural Network Approach
Institute of Climate and Energy Systems, Energy Systems Engineering (ICE-1), Forschungszentrum Jülich GmbH（气候与能源系统研究所，能源系统工程（ICE-1），焦耳中心 GmbH）；RWTH Aachen University（亚琛工业大学）；JARA-ENERGY, Jülich（JARA-ENERGY，焦耳）；RWTH Aachen University, Process Systems Engineering (AVT.SVT)（亚琛工业大学，过程系统工程（AVT.SVT））；RWTH Aachen University, Fluid Process Engineering (AVT.FVT)（亚琛工业大学，流体过程工程（AVT.FVT））；Institute for Bio- and Geosciences (IBG-2), Forschungszentrum Jülich GmbH（生物与地球科学研究所（IBG-2），焦耳中心 GmbH）
Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics
Department of Electronics and Computer Science（电子与计算机科学系）；Mondragon University（蒙得龙大学）；Research & Development（研发）；CARTIF Technology Centre（CARTIF技术中心）；Department of Physical Chemistry（物理化学系）；University of the Basque Country (EHU)（巴斯克国家大学（EHU））；Basque Centre for Applied Mathematics (BCAM)（巴斯克应用数学中心（BCAM））；Department of Computer Science and Artificial Intelligence（计算机科学与人工智能系）
Adaptive Physics-Informed Neural Networks with Multi-Category Feature Engineering for Hydrogen Sorption Prediction in Clays, Shales, and Coals
Faculty of Mathematics & Natural Sciences, University of Oslo（数学与自然科学学院，奥斯陆大学）；SINTEF Industry, Applied Geoscience Department（SINTEF工业公司，应用地球科学部门）；College of Geosciences, China University of Petroleum（地质科学学院，中国石油大学）
Bayesian Physics Informed Neural Networks for Reliable Transformer Prognostics
Department of Electronics & Computing, Mondragon University（蒙龙大学电子与计算系）；Department of Physical Chemistry, University of the Basque Country (UPV/EHU)（巴斯克国家大学物理化学系）；Department of Mathematics, University of the Basque Country (UPV/EHU)（巴斯克国家大学数学系）；Department of Computer Science and Artificial Intelligence, University of the Basque Country (UPV/EHU)（巴斯克国家大学计算机科学与人工智能系）；Basque Centre for Applied Mathematics (BCAM)（巴斯克应用数学中心）；Ikerbasque, Basque Foundation for Science（巴斯克科学基金会）
Most music source separation systems require large collections of isolated sources for training, which can be difficult to obtain. In this work, we use musical scores, which are comparatively easy to obtain, as a weak label for training a source separation system. In contrast with previous score-informed separation approaches, our system does not require isolated sources, and score is used only as a training target, not required for inference. Our model consists of a separator that outputs a time-frequency mask for each instrument, and a transcriptor that acts as a critic, providing both tempo…
${\bf Purpose}$: Earlier work showed that IVIM-NET$_{orig}$, an unsupervised physics-informed deep neural network, was more accurate than other state-of-the-art intravoxel-incoherent motion (IVIM) fitting approaches to DWI. This study presents an improved version: IVIM-NET$_{optim}$, and characterizes its superior performance in pancreatic ductal adenocarcinoma (PDAC) patients. ${\bf Method}$: In simulations (SNR=20), the accuracy, independence and consistency of IVIM-NET were evaluated for combinations of hyperparameters (fit S0, constraints, network architecture, # hidden layers, dropout, ba…
We study the problem of single-channel source separation (SCSS), and focus on cyclostationary signals, which are particularly suitable in a variety of application domains. Unlike classical SCSS approaches, we consider a setting where only examples of the sources are available rather than their models, inspiring a data-driven approach. For source models with underlying cyclostationary Gaussian constituents, we establish a lower bound on the attainable mean squared error (MSE) for any separation method, model-based or data-driven. Our analysis further reveals the operation for optimal separation…
Despite significant progress in object categorization, in recent years, a number of important challenges remain, mainly, ability to learn from limited labeled data and ability to recognize object classes within large, potentially open, set of labels. Zero-shot learning is one way of addressing these challenges, but it has only been shown to work with limited sized class vocabularies and typically requires separation between supervised and unsupervised classes, allowing former to inform the latter but not vice versa. We propose the notion of semi-supervised vocabulary-informed learning to allev…
Music is often experienced as a progression of concurrent streams of notes, or voices. The degree to which this happens depends on the position along a voice-leading continuum, ranging from monophonic, to homophonic, to polyphonic, which complicates the design of automatic voice separation models. We address this continuum by defining voice separation as the task of decomposing music into streams that exhibit both a high degree of external perceptual separation from the other streams and a high degree of internal perceptual consistency. The proposed voice separation task allows for a voice to …
Fudan University（复旦大学）；AITRICS；Hefei University of Technology（合肥工业大学）；University of British Columbia（不列颠哥伦比亚大学）
Despite significant progress in object categorization, in recent years, a number of important challenges remain; mainly, the ability to learn from limited labeled data and to recognize object classes within large, potentially open, set of labels. Zero-shot learning is one way of addressing these challenges, but it has only been shown to work with limited sized class vocabularies and typically requires separation between supervised and unsupervised classes, allowing former to inform the latter but not vice versa. We propose the notion of vocabulary-informed learning to alleviate the above menti…
Separated-Variable Spectral Neural Networks: A Physics-Informed Learning Approach for High-Frequency PDEs
School of Mathematics and Statistics（数学与统计学学院）；Department of Engineering Mechanics（工程力学系）；Department of Aeronautical Engineering（航空工程系）；MIIT Key Laboratory of Dynamics and Control of Complex Systems, Northwestern Polytechnical University（复杂系统动力学与控制国家重点实验室，西北工业大学）；College of Computer Science and Technology（计算机科学与技术学院）；National University of Defense Technology（国防科技大学）；Department of Aerospace Engineering（航空航天工程系）
Finite Volume-Informed Neural Network Framework for 2D Shallow Water Equations: Rugged Loss Landscapes and the Importance of Data Guidance
Department of Civil and Environmental Engineering, Pennsylvania State University, University Park, PA, USA（宾夕法尼亚州立大学土木与环境工程系，大学公园，PA，美国）；Institute for Computational and Data Sciences, Pennsylvania State University, University Park, PA, USA（宾夕法尼亚州立大学计算与数据科学研究所，大学公园，PA，美国）
Fast training of accurate physics-informed neural networks without gradient descent
Technical University of Munich（慕尼黑技术大学）；TUM Institute for Advanced Study（TUM高级研究学院）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Wageningen University & Research（瓦赫宁根大学与研究中心）；KTH Royal Institute of Technology（皇家理工学院）；Munich Data Science Institute（慕尼黑数据科学研究所）
