The performance of ultrasound elastography (USE) heavily depends on the accuracy of displacement estimation. Recently, Convolutional Neural Networks (CNN) have shown promising performance in optical flow estimation and have been adopted for USE displacement estimation. Networks trained on computer vision images are not optimized for USE displacement estimation since there is a large gap between the computer vision images and the high-frequency Radio Frequency (RF) ultrasound data. Many researchers tried to adopt the optical flow CNNs to USE by applying transfer learning to improve the performance of CNNs for USE. However, the ground truth displacement in real ultrasound data is unknown, and simulated data exhibits a domain shift compared to the real data and is also computationally expensive to generate. To resolve this issue, semi-supervised methods have been proposed wherein the networks pre-trained on computer vision images are fine-tuned using real ultrasound data. In this paper, we employ a semi-supervised method by exploiting the first and second-order derivatives of the displacement field for the regularization. We also modify the network structure to estimate both forward and backward displacements, and propose to use consistency between the forward and backward strains as an additional regularizer to further enhance the performance. We validate our method using several experimental phantom and in vivo data. We also show that the network fine-tuned by our proposed method using experimental phantom data performs well on in vivo data similar to the network fine-tuned on in vivo data. Our results also show that the proposed method outperforms current deep learning methods and is comparable to computationally expensive optimization-based algorithms.
Displacement estimation is very important in ultrasound elastography and failing to estimate displacement correctly results in failure in generating strain images. As conventional ultrasound elastography techniques suffer from decorrelation noise, they are prone to fail in estimating displacement between echo signals obtained during tissue distortions. This study proposes a novel elastography technique which addresses the decorrelation in estimating displacement field. We call our method GLUENet (GLobal Ultrasound Elastography Network) which uses deep Convolutional Neural Network (CNN) to get …
Ultrasound elasticity images which enable the visualization of quantitative maps of tissue stiffness can be reconstructed by solving an inverse problem. Classical model-based approaches for ultrasound elastography use deterministic finite element methods (FEMs) to incorporate the governing physical laws resulting in poor performance in noisy conditions. Moreover, these approaches utilize fixed regularizers for various tissue patterns while appropriate data-adaptive priors might be required for capturing the complex spatial elasticity distribution. In this regard, we propose a joint model-based…
Convolutional Neural Networks (CNN) have been found to have great potential in optical flow problems thanks to an abundance of data available for training a deep network. The displacement estimation step in UltraSound Elastography (USE) can be viewed as an optical flow problem. Despite the high performance of CNNs in optical flow, they have been rarely used for USE due to unique challenges that both input and output of USE networks impose. Ultrasound data has much higher high-frequency content compared to natural images. The outputs are also drastically different, where displacement values in …
University of Montreal Hospital Research Center (CRCHUM)（蒙特利尔大学医院研究中心）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California Nanosystems Institute（加州纳米系统研究所）
It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the comp…
Quasi-static ultrasound elastography (USE) is an imaging modality that consists of determining a measure of deformation (i.e.strain) of soft tissue in response to an applied mechanical force. The strain is generally determined by estimating the displacement between successive ultrasound frames acquired before and after applying manual compression. The computational efficiency and accuracy of the displacement prediction, also known as time-delay estimation, are key challenges for real-time USE applications. In this paper, we present a novel deep-learning method for efficient time-delay estimati…
Ultrasound elastography is an emerging noninvasive imaging technique wherein pathological alterations can be visualized by revealing the mechanical properties of the tissue. Estimating tissue displacement in all directions is required to accurately estimate the mechanical properties. Despite capabilities of elastography techniques in estimating displacement in both axial and lateral directions, estimation of axial displacement is more accurate than lateral direction due to higher sampling frequency, higher resolution and having a carrier signal propagating in the axial direction. Among differe…
This paper introduces a novel technique to estimate tissue displacement in quasi-static elastography. A major challenge in elastography is estimation of displacement (also referred to time-delay estimation) between pre-compressed and post-compressed ultrasound data. Maximizing normalized cross correlation (NCC) of ultrasound radio-frequency (RF) data of the pre- and post-compressed images is a popular technique for strain estimation due to its simplicity and computational efficiency. Several papers have been published to increase the accuracy and quality of displacement estimation based on NCC…
Energy-based ultrasound elastography techniques minimize a regularized cost function consisting of data and continuity terms to obtain local displacement estimates based on the local time-delay estimation (TDE) between radio-frequency (RF) frames. The data term associated with the existing techniques takes only the amplitude similarity into account and hence is not sufficiently robust to the outlier samples present in the RF frames under consideration. This drawback creates noticeable artifacts in the strain image. To resolve this issue, we propose to formulate the data function as a linear co…
Model-based computational elasticity imaging of tissues can be posed as solving an inverse problem over finite elements spanning the displacement image. As most existing quasi-static elastography methods count on deterministic formulations of the forward model resulting in a constrained optimization problem, the impact of displacement observation errors has not been well addressed. To this end, we propose a new statistical technique that leads to a unified optimization problem for elasticity imaging. Our statistical model takes the imperfect nature of the displacement measurements into account…
Do We Need Pre-Processing for Deep Learning Based Ultrasound Shear Wave Elastography?
Hamburg University of Technology（汉堡工业大学）；Institute of Medical Technology and Intelligent Systems（医学技术与智能系统研究所）；SustAInLivWork Center of Excellence（SustAInLivWork卓越中心）
Time delay estimation (TDE) is a critical and challenging step in all ultrasound elastography methods. A growing number of TDE techniques require an approximate but robust and fast method to initialize solving for TDE. Herein, we present a fast method for calculating an approximate TDE between two radio frequency (RF) frames of ultrasound. Although this approximate TDE can be useful for several algorithms, we focus on GLobal Ultrasound Elastography (GLUE), which currently relies on Dynamic Programming (DP) to provide this approximate TDE. We exploit Principal Component Analysis (PCA) to find t…
Displacement estimation is a critical step of virtually all Ultrasound Elastography (USE) techniques. Two main features make this task unique compared to the general optical flow problem: the high-frequency nature of ultrasound radio-frequency (RF) data and the governing laws of physics on the displacement field. Recently, the architecture of the optical flow networks has been modified to be able to use RF data. Also, semi-supervised and unsupervised techniques have been employed for USE by considering prior knowledge of displacement continuity in the form of the first- and second-derivative r…
Time delay estimation (TDE) between two radio-frequency (RF) frames is one of the major steps of quasi-static ultrasound elastography, which detects tissue pathology by estimating its mechanical properties. Regularized optimization-based techniques, a prominent class of TDE algorithms, optimize a non-linear energy functional consisting of data constancy and spatial continuity constraints to obtain the displacement and strain maps between the time-series frames under consideration. The existing optimization-based TDE methods often consider the L2-norm of displacement derivatives to construct th…
Shenzhen Research Institute of Big Data, the Chinese University of Hong Kong（香港中文大学深圳大数据研究院）；Huazhong University of Science and Technology（华中科技大学）；Shenzhen Third People’s Hospital（深圳市第三人民医院）
With the development of radiomics, noninvasive diagnosis like ultrasound (US) imaging plays a very important role in automatic liver fibrosis diagnosis (ALFD). Due to the noisy data, expensive annotations of US images, the application of Artificial Intelligence (AI) assisting approaches encounters a bottleneck. Besides, the use of mono-modal US data limits the further improve of the classification results. In this work, we innovatively propose a multi-modal fusion network with active learning (MMFN-AL) for ALFD to exploit the information of multiple modalities, eliminate the noisy data and red…
Concordia University（康考迪亚大学）；University of Montreal（蒙特利尔大学）
Ultrasound elastography is used to estimate the mechanical properties of the tissue by monitoring its response to an internal or external force. Different levels of deformation are obtained from different tissue types depending on their mechanical properties, where stiffer tissues deform less. Given two radio frequency (RF) frames collected before and after some deformation, we estimate displacement and strain images by comparing the RF frames. The quality of the strain image is dependent on the type of motion that occurs during deformation. In-plane axial motion results in high-quality strain…
Ultrasound elastography estimates the mechanical properties of the tissue from two Radio-Frequency (RF) frames collected before and after tissue deformation due to an external or internal force. This work focuses on strain imaging in quasi-static elastography, where the tissue undergoes slow deformations and strain images are estimated as a surrogate for elasticity modulus. The quality of the strain image depends heavily on the underlying deformation, and even the best strain estimation algorithms cannot estimate a good strain image if the underlying deformation is not suitable. Herein, we int…
Ultrasound Elastography aims to determine the mechanical properties of the tissue by monitoring tissue deformation due to internal or external forces. Tissue deformations are estimated from ultrasound radio frequency (RF) signals and are often referred to as time delay estimation (TDE). Given two RF frames I1 and I2, we can compute a displacement image which shows the change in the position of each sample in I1 to a new position in I2. Two important challenges in TDE include high computational complexity and the difficulty in choosing suitable RF frames. Selecting suitable frames is of high im…
Objective: Strain elastography and shear wave elastography are two commonly used methods to quantify cervical elasticity; however, they have limitations. Strain elastography is effective in showing tissue elasticity distribution in a single image, but the absence of stress information causes difficulty in comparing the results acquired from different imaging sessions. Shear wave elastography is effective in measuring shear wave speed (an intrinsic tissue property correlated with elasticity) in relatively homogeneous tissue, such as in the liver. However, for inhomogeneous tissue in the cervix,…
Ultrasound elastography is the method to image the elasticity of compliant tissues due to a mechanical compression applied to it. In elastography, the local strain of explored tissue is estimated by analyzing the echo signals. This is accomplished by a sonographer who uses ultrasound transducer to apply pressure on the tissue area causing displacement of the tissue. A set of data is obtained before and after the compression. This stress along the axial direction not only causes deformation in that direction but it can also cause lateral deformation in the tissue. This relative deformation is l…
Computer-aided detection (CAD) of benign and malignant breast lesions becomes increasingly essential in breast ultrasound (US) imaging. The CAD systems rely on imaging features identified by the medical experts for their performance, whereas deep learning (DL) methods automatically extract features from the data. The challenge of the DL is the insufficiency of breast US images available to train the DL models. Here, we present an ensemble transfer learning model to classify benign and malignant breast tumors using B-mode breast US (B-US) and strain elastography breast US (SE-US) images. This m…
Eindhoven University of Technology（埃因霍温理工大学）；University of Amsterdam（阿姆斯特丹大学）；University Hospital Hamburg-Eppendorf（汉堡-埃彭多夫大学医院）；Universidade de São Paulo Faculdade de Medicina Hospital das Clínicas（圣保罗大学医学院临床医院）
Shear-wave elastography (SWE) permits local estimation of tissue elasticity, an important imaging marker in biomedicine. This recently-developed, advanced technique assesses the speed of a laterally-travelling shear wave after an acoustic radiation force "push" to estimate local Young's moduli in an operator-independent fashion. In this work, we show how synthetic SWE (sSWE) images can be generated based on conventional B-mode imaging through deep learning. Using side-by-side-view B-mode/SWE images collected in 50 patients with prostate cancer, we show that sSWE images with a pixel-wise mean a…
Convolutional Neural Networks (CNN) have shown promising results for displacement estimation in UltraSound Elastography (USE). Many modifications have been proposed to improve the displacement estimation of CNNs for USE in the axial direction. However, the lateral strain, which is essential in several downstream tasks such as the inverse problem of elasticity imaging, remains a challenge. The lateral strain estimation is complicated since the motion and the sampling frequency in this direction are substantially lower than the axial one, and a lack of carrier signal in this direction. In comput…
Department of Internal medicine, National Taiwan University（台湾大学医学院）；department of medicine, national Taiwan University（台湾大学医学院）；department of internal medicine, National Taiwan University Hospital（台湾大学医学院附设医院）；department of medicine, National Taiwan University Cancer center（台湾大学医学院癌症中心）
The purpose of this study is to differentiate malignant and benign mediastinal lesions by using the three-dimensional convolutional neural network through the endobronchial ultrasound (EBUS) image. Compared with previous study, our proposed model is robust to noise and able to fuse various imaging features and spatiotemporal features of EBUS videos. Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is a diagnostic tool for intrathoracic lymph nodes. Physician can observe the characteristics of the lesion using grayscale mode, doppler mode, and elastography during the…
École polytechnique fédérale de Lausanne（洛桑联邦理工学院）；University Hospital Center (CHUV)（大学医院中心（CHUV））；University of Lausanne（洛桑大学）；Center for Biomedical Imaging (CIBM)（生物医学成像中心（CIBM））
Thanks to its capability of acquiring full-view frames at multiple kilohertz, ultrafast ultrasound imaging unlocked the analysis of rapidly changing physical phenomena in the human body, with pioneering applications such as ultrasensitive flow imaging in the cardiovascular system or shear-wave elastography. The accuracy achievable with these motion estimation techniques is strongly contingent upon two contradictory requirements: a high quality of consecutive frames and a high frame rate. Indeed, the image quality can usually be improved by increasing the number of steered ultrafast acquisition…
Non-invasive vascular elastography is an emerging technique in vascular tissue imaging. During the past decades, several techniques have been suggested to estimate the tissue elasticity by measuring the displacement of the Carotid vessel wall. Cross correlation-based methods are the most prevalent approaches to measure the strain exerted in the wall vessel by the blood pressure. In the case of a low pressure, the displacement is too small to be apparent in ultrasound imaging, especially in the regions far from the center of the vessel, causing a high error of displacement measurement. On the o…
Ultrasound Shear Wave Elastography (USWE) with conventional B-mode imaging demonstrates better performance in lesion segmentation and classification problems. In this article, we propose SHEAR-net, an end-to-end deep neural network, to reconstruct USWE images from tracked tissue displacement data at different time instants induced by a single acoustic radiation force (ARF) with 100% or 50% of the energy in conventional use. The SHEAR-net consists of a localizer called the S-net to first localize the lesion location and then uses recurrent layers to extract temporal correlations from wave patte…
École polytechnique fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；University Hospital Center (CHUV)（大学医院中心（CHUV））；University of Lausanne (UNIL)（洛桑大学）；CIBM Center for Biomedical Imaging（CIBM生物医学成像中心）
Ultrafast ultrasound (US) revolutionized biomedical imaging with its capability of acquiring full-view frames at over 1 kHz, unlocking breakthrough modalities such as shear-wave elastography and functional US neuroimaging. Yet, it suffers from strong diffraction artifacts, mainly caused by grating lobes, side lobes, or edge waves. Multiple acquisitions are typically required to obtain a sufficient image quality, at the cost of a reduced frame rate. To answer the increasing demand for high-quality imaging from single unfocused acquisitions, we propose a two-step convolutional neural network (CN…
Ultrasound simulation based on ray tracing enables the synthesis of highly realistic images. It can provide an interactive environment for training sonographers as an educational tool. However, due to high computational demand, there is a trade-off between image quality and interactivity, potentially leading to sub-optimal results at interactive rates. In this work we introduce a deep learning approach based on adversarial training that mitigates this trade-off by improving the quality of simulated images with constant computation time. An image-to-image translation framework is utilized to tr…
Most ultrasound imaging techniques necessitate the fundamental step of converting temporal signals received from transducer elements into a spatial echogenecity map. This beamforming (BF) step requires the knowledge of speed-of-sound (SoS) value in the imaged medium. An incorrect assumption of BF SoS leads to aberration artifacts, not only deteriorating the quality and resolution of conventional brightness mode (B-mode) images, hence limiting their clinical usability, but also impairing other ultrasound modalities such as elastography and spatial SoS reconstructions, which rely on faithfully b…
