Most of the available advanced misfit functions for full waveform inversion (FWI) are hand-crafted, and the performance of those misfit functions is data-dependent. Thus, we propose to learn a misfit function for FWI, entitled ML-misfit, based on machine learning. Inspired by the optimal transport of the matching filter misfit, we design a neural network (NN) architecture for the misfit function in a form similar to comparing the mean and variance for two distributions. To guarantee the resulting learned misfit is a metric, we accommodate the symmetry of the misfit with respect to its input and a Hinge loss regularization term in a meta-loss function to satisfy the "triangle inequality" rule. In the framework of meta-learning, we train the network by running FWI to invert for randomly generated velocity models and update the parameters of the NN by minimizing the meta-loss, which is defined as accumulated difference between the true and inverted models. We first illustrate the basic principle of the ML-misfit for learning a convex misfit function for travel-time shifted signals. Further, we train the NN on 2D horizontally layered models, and we demonstrate the effectiveness and robustness of the learned ML-misfit by applying it to the well-known Marmousi model.
It is challenging for full-waveform inversion to determine geologically informative models from field data. An inaccurate wavelet can make it more complicated. We develop a novel misfit function, entitled deconvolutional double-difference misfit measurement to cancel the influence of wavelet inaccuracy on inversion results. Unlike the popular double-difference misfit measurement in which the first difference is evaluated by cross-correlation, the proposed one employs deconvolution to do this step. Numerical examples demonstrate that full-waveform inversion with the new misfit function is resil…
The data-driven approach has been demonstrated as a promising technique to solve complicated scientific problems. Full Waveform Inversion (FWI) is commonly epitomized as an image-to-image translation task, which motivates the use of deep neural networks as an end-to-end solution. Despite being trained with synthetic data, the deep learning-driven FWI is expected to perform well when evaluated with sufficient real-world data. In this paper, we study such properties by asking: how robust are these deep neural networks and how do they generalize? For robustness, we prove the upper bounds of the d…
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
In the workflow of Full-Waveform Inversion (FWI), we often tune the parameters of the inversion to help us avoid cycle skipping and obtain high resolution models. For example, typically start by using objective functions that avoid cycle skipping, like tomographic and image based or using only low frequency, and then later, we utilize the least squares misfit to admit high resolution information. We also may perform an isotropic (acoustic) inversion to first update the velocity model and then switch to multi-parameter anisotropic (elastic) inversions to fully recover the complex physics. Such …
Conventional full waveform inversion (FWI) using least square distance (LSD) between the observed and predicted seismograms suffers from local minima. Recently, earth mover's distance (EMD) has been introduced to FWI to compute the misfit between two seismograms. Instead of comparisons bin by bin, EMD allows to compare signal intensities across different coordinates. This measure has great potential to account for time and space shifts of events within seismograms. However, there are two main challenges in application of EMD to FWI. The first one is that the compared signals need to satisfy no…
We have formulated elastic seismic full waveform inversion (FWI) within a deep learning environment. In our formulation, a recurrent neural network is set up with rules enforcing elastic wave propagation, with the wavefield projected onto a measurement surface acting as the synthetic data to be compared with observed seismic data. Gradients for iterative updating of an elastic model, with a variety of parameterizations and misfit functionals, can be efficiently constructed within the network through the automatic differential method. With this method, the inversion based on complex misfits can…
Full-waveform inversion is a cutting-edge methodology for recovering high-resolution subsurface models. However, one of the main conventional full-waveform optimization problems challenges is cycle-skipping, usually leading us to an inaccurate local minimum model. A highly investigated track to alleviate this challenge involves designing a more global measure of misfit between the observed and modelled data beyond the sample-to-sample comparison. However, most of these approaches admit relatively smooth inversion results. Here, we introduce a novel misfit function based on the Fourier-based me…
The quantitative reconstruction of sub-surface Earth properties from the propagation of waves follows an iterative minimization of a misfit functional. In marine seismic exploration, the observed data usually consist of measurements of the pressure field but dual-sensor devices also provide the normal velocity. Consequently, a reciprocity-based misfit functional is specifically designed, and defines the Full Reciprocity-gap Waveform Inversion (FRgWI ) method. This misfit functional provides additional features compared to the more traditional least-squares approaches with, in particular, that …
Full-waveform inversion (FWI) is a technique having the potential for building high-resolution elastic velocity models. We proposed to apply this technique to wireline monopole acoustic logging data to obtain the near wellbore formation velocity structures, which can be used in wellbore damage or fluid intrusion evaluation. A 2D FWI using monopole acoustic logging data is presented. The FWI is established in cylindrical coordinates instead of Cartesian coordinates in order to adapt to the borehole geometry. A preconditioner is designed for suppressing the influence of the strong borehole guide…
We introduce a novel approach to waveform inversion, based on a data driven reduced order model (ROM) of the wave operator. The presentation is for the acoustic wave equation, but the approach can be extended to elastic or electromagnetic waves. The data are time resolved measurements of the pressure wave gathered by an acquisition system which probes the unknown medium with pulses and measures the generated waves. We propose to solve the inverse problem of velocity estimation by minimizing the square misfit between the ROM computed from the recorded data and the ROM computed from the modeled …
We propose a multi-model formulation of full-waveform inversion that is similar to image decomposition into a "cartoon" and "texture" used in image processing. Inversion problem is formulated as unconstrained multi-norm optimization that can be solved using conventional iterative solvers. We demonstrate the proposed model decomposition approach by recovering a blocky subsurface seismic model from noisy data in time-lapse and single-model full-waveform inversion problems.
Ambient Noise Full Waveform Inversion with Neural Operators
Seismological Laboratory, California Institute of Technology（加州理工学院地震实验室）；Department of Geology and Geophysics, University of Utah（犹他大学地质与地球物理系）；Nvidia Corporation（英伟达公司）
Seismic full-waveform inversion (FWI) techniques aim to find a high-resolution subsurface geophysical model provided with waveform data. Some recent effort in data-driven FWI has shown some encouraging results in obtaining 2D velocity maps. However, due to high computational complexity and large memory consumption, the reconstruction of 3D high-resolution velocity maps via deep networks is still a great challenge. In this paper, we present InversionNet3D, an efficient and scalable encoder-decoder network for 3D FWI. The proposed method employs group convolution in the encoder to establish an e…
We consider the high-resolution seismic imaging method called full-waveform inversion (FWI). FWI is a data fitting method aimed at inverting for subsurface mechanical parameters. Despite the large adoption of FWI by the academic and industrial communities, and many successful results, FWI still suffers from severe limitations. From a mathematical standpoint, FWI is a large scale PDE-constrained optimization problem. The misfit function that is used, which measures the discrepancy between observed seismic data and data calculated through the solution of a wave propagation problem, is non-convex…
I train a Generative Adversarial Network to produce realistic seismic wave speed models. I integrate the generator network into seismic Full-Waveform Inversion to reduce the number of model parameters and restrict the inverted models to only those that are plausible. Applying the method to a 2D section of the SEAM model, I demonstrate that it can produce more plausible results than conventional Full-Waveform Inversion.
Geophysical models usually contain both sharp interfaces and smooth variations, and it is difficult to accurately account for both of these two types of medium parameter variations using conventional full-waveform inversion methods. In addition, sparse geometry, noisy data and source encoding usually lead to strong inversion artifacts. We develop a novel full-waveform inversion method for acoustic and elastic waves using a total generalized p-variation regularization scheme to address these challenging problems. We decompose the full-waveform inversion into two subproblems and solve these two …
Application of Optimal Transport and the Quadratic Wasserstein Metric to Full-Waveform Inversion
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Bureau of Economic Geology, John A. and Katherine G. Jackson School of Geosciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校杰克逊地球科学学院经济地质局）；New Jersey Institute of Technology（新泽西理工学院）
We propose and test the Direct Waveform Inversion (DWI) scheme to simultaneously invert for layered velocity and density profiles, using reflection seismic waveforms recorded on the surface. The recorded data include primary reflections and interbed multiples. DWI is implemented in the time-space domain. By explicitly enforcing the wavefield time-space causality, DWI can recursively determine the subsurface seismic structure in a local fashion for both sharp interfaces as well as the properties of the layers, from shallow to deep depths. DWI is different from the layer stripping methods in the…
The dynamic time warping (DTW) distance has been used as a misfit function for wave-equation inversion to mitigate the local minima issue. However, the original DTW distance is not smooth; therefore it can yield a strong discontinuity in the adjoint source. Such a weakness does not help nonlinear inverse problems converge to a plausible minimum by any means. We therefore introduce for the first time in geophysics the smooth DTW distance, which has demonstrated its performance in time series classification, clustering, and prediction as the loss function. The fundamental idea of such a distance…
Airborne transient electromagnetic (TEM) is a cost-effective method to image the distribution of electrical conductivity in the ground. We consider layered earth inversion to interpret large data sets of hundreds of kilometre. Different strategies can be used to solve this inverse problem. This consists in managing the a priori information to avoid the mathematical instability and provide the most plausible model of conductivity in depth. In order to obtain fast and realistic inversion program, we tested three kinds of regularization: two are based on standard Tikhonov procedure which consist …
We present a technique for reconstructing subsurface velocity model changes from time-lapse seismic survey data using full-waveform inversion (FWI). The technique is based on simultaneously inverting multiple survey vintages, with model difference regularization using the total variation (TV) seminorm. We compare the new TV-regularized time-lapse FWI with the $L_2$-regularized joint inversion proposed in our earlier work, using synthetic data sets that exhibit survey repeatability issues. The results demonstrate clear advantages of the proposed TV-regularized joint inversion over alternatives …
We propose two preconditioned gradient direction for full waveform inversion (FWI). The first one is using time integral wavefields. The Least square problem is formulated as the time integral residual wavefields, which can partially resolve the effect of high-passed filter in the traditional gradient formula; the convergence rate is greatly accelerated. The other one is localized offset Hessian inspired by the generalized imaging condition, which provides another redundancy in the Hessian. We compare the traditional conjugate gradient scaled by the shot illumination and localized offset Hessi…
RED-DiffEq: Regularization by denoising diffusion models for solving inverse PDE problems with application to full waveform inversion
Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）；School of Data Science and Society, University of North Carolina at Chapel Hill（北卡罗来纳大学夏洛特分校数据科学与社会学院）
Traditional seismic envelope inversion takes use of a nonlinear misfit functional which relates the envelope of seismogram to the observed wavefield records, and then derive the sensitivity kernel of envelope to velocity through the use of waveform Frechet derivative by linearizing the nonlinear functional. We know that the waveform Frechet derivative is based on the Born approximation of wave scattering and can only work for the case of weak scattering. Therefore, the traditional envelope inversion using waveform Frechet derivative has severe limitation in the case of strong-scattering for la…
Full waveform inversion (FWI) aims at estimating subsurface medium properties from measured seismic data. It is usually cast as a non-linear least-squares problem that incorporates uncertainties in the measurements. In exploration seismology, extended formulations of FWI that allow for uncertaties in the physics have been proposed. Even when the physics is modelled accurately, these extensions have been shown to be beneficial because they reduce the non-lineary of the resulting data-fitting problem. In this note, I derive an alternative (but equivalent) formulation of extended full waveform in…
Full waveform inversion (FWI) is a powerful yet computationally expensive technique that can yield subsurface models at high resolution. Randomly selected shots ("mini-batches") can be used to approximate the misfit and the gradient of FWI, thereby reducing its computational cost. Here, we present a methodology to perform mini-batch FWI using the Adam algorithm, an adaptive optimization scheme based on stochastic gradient descent. It provides for stable model updates by smoothing the gradient across iterations and can also account for the curvature of the optimization landscape. We describe em…
PreAdaptFWI: Pretrained-Based Adaptive Residual Learning for Full-Waveform Inversion Without Dataset Dependency
College of Instrumentation and Electrical Engineering, Jilin University（吉林大学仪器科学与电气工程学院）；Northeast Electric Power University（东北电力大学）；College of Communication Engineering, Jilin University（吉林大学通信工程学院）
I demonstrate that the conventional seismic full-waveform inversion algorithm can be constructed as a recurrent neural network and so implemented using deep learning software such as TensorFlow. Applying another deep learning concept, the Adam optimizer with minibatches of data, produces quicker convergence toward the true wave speed model on a 2D dataset than Stochastic Gradient Descent and than the L-BFGS-B optimizer with the cost function and gradient computed using the entire training dataset. I also show that the cost function gradient calculation using reverse-mode automatic differentiat…
We propose a formulation of full-wavefield inversion (FWI) as a constrained optimization problem, and describe a computationally efficient technique for solving constrained full-wavefield inversion (CFWI). The technique is based on using a total-variation regularization method, with the regularization weighted in favor of constraining deeper subsurface model sections. The method helps to promote "edge-preserving" blocky model inversion where fitting the seismic data alone fails to adequately constrain the model. The method is demonstrated on synthetic datasets with added noise, and is shown to…
Seismic full-waveform inversion (FWI) uses full seismic records to estimate subsurface velocity structure. This requires a highly nonlinear and nonunique inverse problem to be solved, and Bayesian methods have been used to quantify uncertainties in the solution. Variational Bayesian inference uses optimization to provide solutions efficiently. The method has been applied to solve a transmission FWI problem using data generated by known earthquake-like sources, with strong prior information imposed on the velocity. Unfortunately such prior information about velocity structure and earthquake sou…
In seismic waveform inversion, the reconstruction of the subsurface properties is usually carried out using approximative wave propagation models to ensure computational efficiency. The viscoelastic nature of the subsurface is often unaccounted for, and two popular approximations--the acoustic and linearized Born inversion--are widely used. This leads to reconstruction errors since the approximations ignore realistic (physical) aspects of seismic wave propagation in the heterogeneous earth. In this study, we show that the Bayesian approximation error approach can be used to partially recover f…
Tensor Train Decomposition-based 3D Implicit Full Waveform Inversion with Multi-scale Structural Similarity
State Key Laboratory of Deep Earth Exploration and Imaging（深部地球探测与成像国家重点实验室）；College of GeoExploration Science and Technology, Jilin University（吉林大学地球探测科学与技术学院）；Petroleum Exploration and Production Research Institute, Sinopec（中石油石油勘探生产研究院）
Non-invasive subsurface imaging using full waveform inversion (FWI) has the potential to fundamentally change engineering site characterization by enabling the recovery of high resolution 2D/3D maps of subsurface stiffness. Yet, the accuracy of FWI remains quite sensitive to the choice of the initial starting model due to the complexity and non-uniqueness of the inverse problem. In response, we present the novel application of convolutional neural networks (CNNs) to transform an experimental seismic wavefield acquired using a linear array of surface sensors directly into a robust starting mode…
Full-waveform inversion (FWI) is an accurate imaging approach for modeling velocity structure by minimizing the misfit between recorded and predicted seismic waveforms. However, the strong non-linearity of FWI resulting from fitting oscillatory waveforms can trap the optimization in local minima. We propose a neural-network-based full waveform inversion method (NNFWI) that integrates deep neural networks with FWI by representing the velocity model with a generative neural network. Neural networks can naturally introduce spatial correlations as regularization to the generated velocity model, wh…
Universidade Federal do Espírito Santo（巴西联邦圣埃斯皮里图大学）；Instituto Federal do Espírito Santo（圣埃斯皮里图联邦理工学院）；Exploração e Produção Petróleo Brasileiro S.A.（巴西国家石油公司勘探与生产部门）
In the Oil and Gas industry, estimating a subsurface velocity field is an essential step in seismic processing, reservoir characterization, and hydrocarbon volume calculation. Full-waveform inversion (FWI) velocity modeling is an iterative advanced technique that provides an accurate and detailed velocity field model, although at a very high computational cost due to the physics-based numerical simulations required at each FWI iteration. In this study, we propose a method of generating velocity field models, as detailed as those obtained through FWI, using a conditional generative adversarial …
Full-Waveform Inversion (FWI) has now become a widely accepted tool to obtain high-resolution velocity models from seismic data. Typically, the velocity model in its discrete form is represented on a rectangular grid, and we solve for the elastic properties at these grid points. FWI is mostly solved employing a local optimization method, where one obtains a velocity update by minimizing the misfit between the observed and the calculated seismograms. Note also that FWI is a highly non-linear problem which is known to be prone to non-uniqueness. The convergence to a globally optimum solution is …
Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；The Pennsylvania State University（宾夕法尼亚州立大学）；Michigan State University（密歇根州立大学）；Microsoft Research（微软研究院）
This paper investigates unsupervised learning of Full-Waveform Inversion (FWI), which has been widely used in geophysics to estimate subsurface velocity maps from seismic data. This problem is mathematically formulated by a second order partial differential equation (PDE), but is hard to solve. Moreover, acquiring velocity map is extremely expensive, making it impractical to scale up a supervised approach to train the mapping from seismic data to velocity maps with convolutional neural networks (CNN). We address these difficulties by integrating PDE and CNN in a loop, thus shifting the paradig…
We describe a new method, full waveform inversion by model extension (FWIME) that recovers accurate acoustic subsurface velocity models from seismic data, when conventional methods fail. We leverage the advantageous convergence properties of wave-equation migration velocity analysis (WEMVA) with the accuracy and high-resolution nature of acoustic full waveform inversion (FWI) by combining them into a robust mathematically-consistent workflow with minimal need for user inputs. The novelty of FWIME resides in the design of a new cost function using the variable projection method, and a novel opt…
