Uncertainty is ubiquitous with flow in subsurface rocks because of their inherent heterogeneity and lack of in-situ measurements. To complete uncertainty analysis in a multi-scale manner, it is a prerequisite to provide sufficient rock samples. Even though the advent of digital rock technology offers opportunities to reproduce rocks, it still cannot be utilized to provide massive samples due to its high cost, thus leading to the development of diversified mathematical methods. Among them, two-point statistics (TPS) and multi-point statistics (MPS) are commonly utilized, which feature incorporating low-order and high-order statistical information, respectively. Recently, generative adversarial networks (GANs) are becoming increasingly popular since they can reproduce training images with excellent visual and consequent geologic realism. However, standard GANs can only incorporate information from data, while leaving no interface for user-defined properties, and thus may limit the representativeness of reconstructed samples. In this study, we propose conditional GANs for digital rock reconstruction, aiming to reproduce samples not only similar to the real training data, but also satisfying user-specified properties. In fact, the proposed framework can realize the targets of MPS and TPS simultaneously by incorporating high-order information directly from rock images with the GANs scheme, while preserving low-order counterparts through conditioning. We conduct three reconstruction experiments, and the results demonstrate that rock type, rock porosity, and correlation length can be successfully conditioned to affect the reconstructed rock images. Furthermore, in contrast to existing GANs, the proposed conditioning enables learning of multiple rock types simultaneously, and thus invisibly saves computational cost.
Peng Cheng Laboratory（鹏城实验室）；Southern University of Science and Technology（南方科技大学）
Random reconstruction of three-dimensional (3D) digital rocks from two-dimensional (2D) slices is crucial for elucidating the microstructure of rocks and its effects on pore-scale flow in terms of numerical modeling, since massive samples are usually required to handle intrinsic uncertainties. Despite remarkable advances achieved by traditional process-based methods, statistical approaches and recently famous deep learning-based models, few works have focused on producing several kinds of rocks with one trained model and allowing the reconstructed samples to satisfy certain given properties, s…
University of New South Wales（新南威尔士大学）；School of Minerals and Energy Resources Engineering（矿物与能源资源工程学院）
Digital Rock Imaging is constrained by detector hardware, and a trade-off between the image field of view (FOV) and the image resolution must be made. This can be compensated for with super resolution (SR) techniques that take a wide FOV, low resolution (LR) image, and super resolve a high resolution (HR), high FOV image. The Enhanced Deep Super Resolution Generative Adversarial Network (EDSRGAN) is trained on the Deep Learning Digital Rock Super Resolution Dataset, a diverse compilation 12000 of raw and processed uCT images. The network shows comparable performance of 50% to 70% reduction in …
Each grid block in a 3D geological model requires a rock type that represents all physical and chemical properties of that block. The properties that classify rock types are lithology, permeability, and capillary pressure. Scientists and engineers determined these properties using conventional laboratory measurements, which embedded destructive methods to the sample or altered some of its properties (i.e., wettability, permeability, and porosity) because the measurements process includes sample crushing, fluid flow, or fluid saturation. Lately, Digital Rock Physics (DRT) has emerged to quantif…
Cavendish Laboratory（卡文迪什实验室）；University of Cambridge（剑桥大学）
Holographic wave-shaping has found numerous applications across the physical sciences, especially since the development of digital spatial-light modulators (SLMs). A key challenge in digital holography consists in finding optimal hologram patterns which transform the incoming laser beam into desired shapes in a conjugate optical plane. The existing repertoire of approaches to solve this inverse problem is built on iterative phase-retrieval algorithms, which do not take optical aberrations and deviations from theoretical models into account. Here, we adopt a physics-free, data-driven, and proba…
Large Intestine 3D Shape Refinement Using Point Diffusion Models for Digital Phantom Generation
Center for Virtual Imaging Trials, Department of Radiology, Duke University School of Medicine, Durham, NC, USA Biomedical Imaging Group Rotterdam, Department of Radiology \& Nuclear Medicine, Erasmus MC, Rotterdam, the Netherlands Electrical；Computer Engineering, Pratt School of Engineering, Duke University, Durham, NC, USA
Estimating Atmospheric Variables from Digital Typhoon Satellite Images via Conditional Denoising Diffusion Models
Imperial College London（帝国理工学院）；University of Cambridge（剑桥大学）；Grantham Institute for Climate Change and the Environment, Imperial College London（帝国理工学院格兰瑟姆气候变化与环境研究所）；National Center for Artificial Intelligence CENIA, Chile（智利国家人工智能中心CENIA）
Tongji Hospital, Huazhong University of Science and Technology（华中科技大学同济医院）
Multi-focus image fusion technologies compress different focus depth images into an image in which most objects are in focus. However, although existing image fusion techniques, including traditional algorithms and deep learning-based algorithms, can generate high-quality fused images, they need multiple images with different focus depths in the same field of view. This criterion may not be met in some cases where time efficiency is required or the hardware is insufficient. The problem is especially prominent in large-size whole slide images. This paper focused on the multi-focus image fusion …
When trained on multimodal image datasets, normal Generative Adversarial Networks (GANs) are usually outperformed by class-conditional GANs and ensemble GANs, but conditional GANs is restricted to labeled datasets and ensemble GANs lack efficiency. We propose a novel GAN variant called virtual conditional GAN (vcGAN) which is not only an ensemble GAN with multiple generative paths while adding almost zero network parameters, but also a conditional GAN that can be trained on unlabeled datasets without explicit clustering steps or objectives other than the adversary loss. Inside the vcGAN's gene…
Toward AI-Driven Digital Twins for Metropolitan Floods: A Conditional Latent Dynamics Network Surrogate of the Shallow Water Equations
School of Computational Science and Engineering, Georgia Institute of Technology（计算科学与工程学院，佐治亚理工学院）；Environmental Science Division, Argonne National Laboratory（环境科学部，阿贡国家实验室）
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）
We present a research study aimed at testing of applicability of machine learning techniques for prediction of permeability of digitized rock samples. We prepare a training set containing 3D images of sandstone samples imaged with X-ray microtomography and corresponding permeability values simulated with Pore Network approach. We also use Minkowski functionals and Deep Learning-based descriptors of 3D images and 2D slices as input features for predictive model training and prediction. We compare predictive power of various feature sets and methods. The later include Gradient Boosting and vario…
A generative adversarial network optimization method for damage detection and digital twinning by deep AI fault learning: Z24 Bridge structural health monitoring benchmark validation
University of Bath（巴斯大学）；Loughborough University（拉夫堡大学）
In this work, we propose a novel generative model for mapping inputs to structured, high-dimensional outputs using structured conditional normalizing flows and Gaussian process regression. The model is motivated by the need to characterize uncertainty in the input/output relationship when making inferences on new data. In particular, in the physical sciences, limited training data may not adequately characterize future observed data; it is critical that models adequately indicate uncertainty, particularly when they may be asked to extrapolate. In our proposed model, structured conditional norm…
Samsung R&D Institute China Beijing（三星中国研发中心（北京））；Beihang University（北京航空航天大学）
Recently, realistic image generation using deep neural networks has become a hot topic in machine learning and computer vision. Images can be generated at the pixel level by learning from a large collection of images. Learning to generate colorful cartoon images from black-and-white sketches is not only an interesting research problem, but also a potential application in digital entertainment. In this paper, we investigate the sketch-to-image synthesis problem by using conditional generative adversarial networks (cGAN). We propose the auto-painter model which can automatically generate compati…
Purdue University（普渡大学）；Politecnico di Milano（米兰理工大学）
The widespread availability of image editing tools and improvements in image processing techniques allow image manipulation to be very easy. Oftentimes, easy-to-use yet sophisticated image manipulation tools yields distortions/changes imperceptible to the human observer. Distribution of forged images can have drastic ramifications, especially when coupled with the speed and vastness of the Internet. Therefore, verifying image integrity poses an immense and important challenge to the digital forensic community. Satellite images specifically can be modified in a number of ways, including the ins…
Universiti Malaysia Sarawak（马来西亚砂拉越大学）；Jabatan Mineral dan Geosains（矿物与地球科学局）；Kementerian Sumber Asli dan Alam Sekitar（自然资源与环境部）
Classification of rocks is one of the fundamental tasks in a geological study. The process requires a human expert to examine sampled thin section images under a microscope. In this study, we propose a method that uses microscope automation, digital image acquisition, edge detection and colour analysis (histogram). We collected 60 digital images from 20 standard thin sections using a digital camera mounted on a conventional microscope. Each image is partitioned into a finite number of cells that form a grid structure. Edge and colour profile of pixels inside each cell determine its classificat…
Flow-based conditional cardiac anatomy generation for virtual cohorts
Delft University of Technology（代尔夫特理工大学）；Hospital of the University of Pennsylvania（宾夕法尼亚大学医院）；Ghent University（根特大学）；University of Pennsylvania（宾夕法尼亚大学）
School of Minerals and Energy Resources Engineering, The University of New South Wales（新南威尔士大学矿物与能源资源工程学院）；CSIRO Energy（澳大利亚联邦科学与工业研究组织能源研究所）
X-ray micro-computed tomography (micro-CT) has been widely leveraged to characterise pore-scale geometry in subsurface porous rock. Recent developments in super resolution (SR) methods using deep learning allow the digital enhancement of low resolution (LR) images over large spatial scales, creating SR images comparable to the high resolution (HR) ground truth. This circumvents traditional resolution and field-of-view trade-offs. An outstanding issue is the use of paired (registered) LR and HR data, which is often required in the training step of such methods but is difficult to obtain. In thi…
Controlled Latent Diffusion Models for 3D Porous Media Reconstruction
Department of Mechanical Engineering, Coppe, Federal University of Rio de Janeiro（机械工程系，Coppe，里约热内卢联邦大学）；Institute of Mathematics, Federal University of Rio de Janeiro（数学研究所，里约热内卢联邦大学）；Department of Geology, Institute of Geosciences, Federal University of Rio de Janeiro（地质系，地球科学研究所，里约热内卢联邦大学）
The prime goal of digital imaging techniques is to reproduce the realistic appearance of a scene. Low Dynamic Range (LDR) cameras are incapable of representing the wide dynamic range of the real-world scene. The captured images turn out to be either too dark (underexposed) or too bright (overexposed). Specifically, saturation in overexposed regions makes the task of reconstructing a High Dynamic Range (HDR) image from single LDR image challenging. In this paper, we propose a deep learning based approach to recover details in the saturated areas while reconstructing the HDR image. We formulate …
A framework is proposed for generative models as a basis for digital twins or mirrors of structures. The proposal is based on the premise that deterministic models cannot account for the uncertainty present in most structural modelling applications. Two different types of generative models are considered here. The first is a physics-based model based on the stochastic finite element (SFE) method, which is widely used when modelling structures that have material and loading uncertainties imposed. Such models can be calibrated according to data from the structure and would be expected to outperf…
Single Image Super Resolution (SISR) techniques based on Super Resolution Convolutional Neural Networks (SRCNN) are applied to micro-computed tomography (μCT) images of sandstone and carbonate rocks. Digital rock imaging is limited by the capability of the scanning device resulting in trade-offs between resolution and field of view, and super resolution methods tested in this study aim to compensate for these limits. SRCNN models SR-Resnet, Enhanced Deep SR (EDSR), and Wide-Activation Deep SR (WDSR) are used on the Digital Rock Super Resolution 1 (DRSRD1) Dataset of 4x downsampled images, comp…
Technion - IIT（以色列理工学院）；Rambam Health Care Campus（兰巴姆医疗中心）
Artificial intelligence and machine learning techniques have the promise to revolutionize the field of digital pathology. However, these models demand considerable amounts of data, while the availability of unbiased training data is limited. Synthetic images can augment existing datasets, to improve and validate AI algorithms. Yet, controlling the exact distribution of cellular features within them is still challenging. One of the solutions is harnessing conditional generative adversarial networks that take a semantic mask as an input rather than a random noise. Unlike other domains, outlining…
Ahsanullah University of Science and Technology（阿赫桑乌拉科技大学）
Jamdani is the strikingly patterned textile heritage of Bangladesh. The exclusive geometric motifs woven on the fabric are the most attractive part of this craftsmanship having a remarkable influence on textile and fine art. In this paper, we have developed a technique based on the Generative Adversarial Network that can learn to generate entirely new Jamdani patterns from a collection of Jamdani motifs that we assembled, the newly formed motifs can mimic the appearance of the original designs. Users can input the skeleton of a desired pattern in terms of rough strokes and our system finalizes…
From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketching
Huzhou University（湖州大学）；Singapore University of Technology and Design（新加坡科技设计大学）；University of Science and Technology of China（中国科学技术大学）；Moxin (Huzhou) Technology Co., LTD.（魔芯（湖州）科技有限公司）；Zhejiang University（浙江大学）
本文提出由 AR/VR 中 3D 草图驱动的服装生成框架，结合条件扩散模型、共享潜空间草图编码器与课程学习，并构建 KO3DClothes 数据集，显著提升数字服装设计的保真度和易用性。
VQ-DSC-R: Robust Vector Quantized-Enabled Digital Semantic Communication With OFDM Transmission
ZGC Institute of Ubiquitous-X Innovation and Applications（ubiquitous-X 创新与应用研究所）；Beijing Key Laboratory of 6G DOICT converged and Cloud-Native Mobile Information Networks（北京6G DOICT融合与云原生移动信息网络重点实验室）；State Key Laboratory of Networking and Switching Technology（网络与交换技术国家重点实验室）；Beijing University of Posts and Telecommunications（北京邮电大学）
Generative modelling powered by room-temperature polariton condensates
School of Mathematical and Physical Sciences, University of Sheffield, Sheffield S10 2TN, United Kingdom（谢菲尔德大学数学与物理科学学院）；Department of Physics, City College of New York, New York, NY 10031, USA（纽约城市学院物理系）；Physics Doctoral Program, Graduate Center of the City University of New York, New York, NY 10016, USA（纽约城市大学研究生中心物理博士项目）；Chemistry Doctoral Program, Graduate Center of the City University of New York, New York, NY 10016, USA（纽约城市大学研究生中心化学博士项目）
University of New South Wales（新南威尔士大学）；Australian National University（澳大利亚国立大学）；School of Minerals and Energy Resources Engineering（矿物与能源资源工程学院）
Segmentation of 3D micro-Computed Tomographic uCT) images of rock samples is essential for further Digital Rock Physics (DRP) analysis, however, conventional methods such as thresholding, watershed segmentation, and converging active contours are susceptible to user-bias. Deep Convolutional Neural Networks (CNNs) have produced accurate pixelwise semantic segmentation results with natural images and $μ$CT rock images, however, physical accuracy is not well documented. The performance of 4 CNN architectures is tested for 2D and 3D cases in 10 configurations. Manually segmented uCT images of Mt. …
Tabular data in digital documents is widely used to express compact and important information for readers. However, it is challenging to parse tables from unstructured digital documents, such as PDFs and images, into machine-readable format because of the complexity of table structures and the missing of meta-information. Table Structure Recognition (TSR) problem aims to recognize the structure of a table and transform the unstructured tables into a structured and machine-readable format so that the tabular data can be further analysed by the down-stream tasks, such as semantic modeling and in…
Remote Sensing Technology Institute, German Aerospace Center（德国航空航天中心遥感技术研究所）；Signal Processing in Earth Observation, Technical University of Munich（慕尼黑工业大学地球观测信号处理部门）
Object retrieval and reconstruction from very high resolution (VHR) synthetic aperture radar (SAR) images are of great importance for urban SAR applications, yet highly challenging owing to the complexity of SAR data. This paper addresses the issue of individual building segmentation from a single VHR SAR image in large-scale urban areas. To achieve this, we introduce building footprints from GIS data as complementary information and propose a novel conditional GIS-aware network (CG-Net). The proposed model learns multi-level visual features and employs building footprints to normalize the fea…
