Modern-day display systems demand high-quality rendering. However, rendering at higher resolution requires a large number of data samples and is computationally expensive. Recent advances in deep learning-based image and video super-resolution techniques motivate us to investigate such networks for high-fidelity upscaling of frames rendered at a lower resolution to a higher resolution. While our work focuses on super-resolution of medical volume visualization performed with direct volume rendering, it is also applicable for volume visualization with other rendering techniques. We propose a learning-based technique where our proposed system uses color information along with other supplementary features gathered from our volume renderer to learn efficient upscaling of a low-resolution rendering to a higher-resolution space. Furthermore, to improve temporal stability, we also implement the temporal reprojection technique for accumulating history samples in volumetric rendering.
The benefits of medical imaging are enormous. Medical images provide considerable amounts of anatomical information and this facilitates medical practitioners in performing effective disease diagnosis and deciding upon the best course of medical treatment. A transition from traditional monochromatic medical images like CT scans, X-Rays or MRI images to a colored 3D representation of the anatomical structure further enhances the capabilities of medical professionals in extracting valuable medical information. The proposed framework in our research starts with performing color transfer by findin…
Analyzing the effects of ocean eddies is important in oceanology for gaining insights into transport of energy and biogeochemical particles. We present an application of statistical visualization algorithms for the analysis of the Red Sea eddy simulation ensemble. Specifically, we demonstrate the applications of statistical volume rendering and statistical Morse complex summary maps to a velocity magnitude field for studying the eddy positions in the flow dataset. In statistical volume rendering, we model per-voxel data uncertainty using noise models, such as parametric and nonparametric, and …
We are living in the big data age: An ever increasing amount of data is being produced through data acquisition and computer simulations. While large scale analysis and simulations have received significant attention for cloud and high-performance computing, software to efficiently visualise large data sets is struggling to keep up. Visualization has proven to be an efficient tool for understanding data, in particular visual analysis is a powerful tool to gain intuitive insight into the spatial structure and relations of 3D data sets. Large-scale visualization setups are becoming ever more aff…
Compared with full volume rendering, isosurface rendering has several well recognized advantages in efficiency and accuracy. However, standard isosurface rendering has some limitations in effectiveness. First, it uses a monotone colored approach and can only visualize the geometry features of an isosurface. The lack of the capability to illustrate the material property and the internal structures behind an isosurface has been a big limitation of this method in applications. Another limitation of isosurface rendering is the difficulty to reveal physically meaningful structures, which are hidden…
Direct Volume Rendering is a popular and powerful visualization method for voxel data and other volumetric scalar data sets. Particularly, in medical applications volume rendering is very commonly used, and has become one of the state of the art methods for 3D visualization of medical data. In this article, some of the most common artifacts encountered will be discussed, and their possible remedies.
We introduce Artifact-Based Rendering (ABR), a framework of tools, algorithms, and processes that makes it possible to produce real, data-driven 3D scientific visualizations with a visual language derived entirely from colors, lines, textures, and forms created using traditional physical media or found in nature. A theory and process for ABR is presented to address three current needs: (i) designing better visualizations by making it possible for non-programmers to rapidly design and critique many alternative data-to-visual mappings; (ii) expanding the visual vocabulary used in scientific visu…
Visualizing regional-scale landslides is essential to conveying the threat of natural hazards to stakeholders and policymakers. Traditional visualization techniques are restricted to post-processing a limited subset of simulation data and are not scalable to rendering regional-scale models. In situ visualization is a technique of rendering simulation data in real-time, i.e., rendering visuals in tandem while the simulation is running. This study develops a scalable N:M interface architecture to visualize regional-scale landslides. We demonstrate the scalability of the architecture by simulatin…
In this paper, we first give a high-level overview of medical visualization development over the past 30 years, focusing on key developments and the trends that they represent. During this discussion, we will refer to a number of key papers that we have also arranged on the medical visualization research timeline. Based on the overview and our observations of the field, we then identify and discuss the medical visualization research challenges that we foresee for the coming decade.
This paper presents a framework that fully leverages the advantages of a deferred rendering approach for the interactive visualization of large-scale datasets. Geometry buffers (G-Buffers) are generated and stored in situ, and shading is performed post hoc in an interactive image-based rendering front end. This decoupled framework has two major advantages. First, the G-Buffers only need to be computed and stored once---which corresponds to the most expensive part of the rendering pipeline. Second, the stored G-Buffers can later be consumed in an image-based rendering front end that enables use…
We present here the result of continuation work, performed to further fulfill the vision we outlined in [Harel,Lekien,Pébaÿ-2017] for the visualization and analysis of tree-based adaptive mesh refinement (AMR) simulations, using the hypertree grid paradigm which we proposed. The first filter presented hereafter implements an adaptive approach in order to accelerate the rendering of 2-dimensional AMR grids, hereby solving the problem posed by the loss of interactivity that occurs when dealing with large and/or deeply refined meshes. Specifically, view parameters are taken into account, in order…
The goal of 3D visualization is to provide the user with an intuitive interface which enables him to explore the 3D data in an interactive manner. The aim of the exploration is to identify and analyze anomalies or to give proof of the non-anomaly of the visualized organic structures. For 3D Medical Data, Magnetic Resonance Images (MRI) has been used. To create the 3D model, we used the Direct Volume Rendering technique. In the input 3D data, we have $x, y$ and $z$ coordinates and an intensity value for each voxel. The 3D data is used by Volume Ray Casting to compute 2D projections from 3D volu…
Volume data is found in many important scientific and engineering applications. Rendering this data for visualization at high quality and interactive rates for demanding applications such as virtual reality is still not easily achievable even using professional-grade hardware. We introduce FoVolNet -- a method to significantly increase the performance of volume data visualization. We develop a cost-effective foveated rendering pipeline that sparsely samples a volume around a focal point and reconstructs the full-frame using a deep neural network. Foveated rendering is a technique that prioriti…
Medical visualization is the use of computers to create 3D images from medical imaging data sets, almost all surgery and cancer treatment in the developed world relies on it.Volume visualization techniques includes iso-surface visualization, mesh visualization and point cloud visualization techniques, these techniques have revolutionized medicine. Much of modern medicine relies on the 3D imaging that is possible with magnetic resonance imaging (MRI) scanners, functional magnetic resonance imaging (fMRI)scanners, positron emission tomography (PET) scanners, ultrasound imaging (US) scanners, X-R…
A recent development, called isogeometric analysis, provides a unified approach for design, analysis and optimization of functional products in industry. Traditional volume rendering methods for inspecting the results from the numerical simulations cannot be applied directly to isogeometric models. We present a novel approach for interactive visualization of isogeometric analysis results, ensuring correct, i.e., pixel-accurate geometry of the volume including its bounding surfaces. The entire OpenGL pipeline is used in a multi-stage algorithm leveraging techniques from surface rendering, order…
Rendering an accurate image of an isosurface in a volumetric field typically requires large numbers of data samples. Reducing the number of required samples lies at the core of research in volume rendering. With the advent of deep learning networks, a number of architectures have been proposed recently to infer missing samples in multi-dimensional fields, for applications such as image super-resolution and scan completion. In this paper, we investigate the use of such architectures for learning the upscaling of a low-resolution sampling of an isosurface to a higher resolution, with high fideli…
Multi-Layer Gaussian Splatting for Immersive Anatomy Visualization
Technical University of Munich（慕尼黑技术大学）；Clinic for Orthopedics and Sports Orthopedics（骨科与运动骨科诊所）；TUM University Hospital（慕尼黑技术大学医院）；Siemens Healthineers AG（西门子医疗影像有限公司）
Viewpoint estimation from 2D rendered images is helpful in understanding how users select viewpoints for volume visualization and guiding users to select better viewpoints based on previous visualizations. In this paper, we propose a viewpoint estimation method based on Convolutional Neural Networks (CNNs) for volume visualization. We first design an overfit-resistant image rendering pipeline to generate the training images with accurate viewpoint annotations, and then train a category-specific viewpoint classification network to estimate the viewpoint for the given rendered image. Our method …
Visualization of large vector line data is a core task in geographic and cartographic systems. Vector maps are often displayed at different cartographic generalization levels, traditionally by using several discrete levels-of-detail (LODs). This limits the generalization levels to a fixed and predefined set of LODs, and generally does not support smooth LOD transitions. However, fast GPUs and novel line rendering techniques can be exploited to integrate dynamic vector map LOD management into GPU-based algorithms for locally-adaptive line simplification and real-time rendering. We propose a new…
Background: Visualization of multi-channel microscopy data plays a vital role in biological research. With the ever-increasing resolution of modern microscopes the data set size of the scanned specimen grows steadily. On commodity hardware this size easily exceeds the available main memory and the even more limited GPU memory. Common volume rendering techniques require the entire data set to be present in the GPU memory. Existing out-of-core rendering approaches for large volume data sets either are limited to single-channel volumes, or require a computer cluster, or have long preprocessing ti…
University of Sarajevo（萨拉热窝大学）；Faculty of Electrical Engineering（电气工程学院）；University of Paderborn（帕德博恩大学）
For medical volume visualization, one of the most important tasks is to reveal clinically relevant details from the 3D scan (CT, MRI ...), e.g. the coronary arteries, without obscuring them with less significant parts. These volume datasets contain different materials which are difficult to extract and visualize with 1D transfer functions based solely on the attenuation coefficient. Multi-dimensional transfer functions allow a much more precise classification of data which makes it easier to separate different surfaces from each other. Unfortunately, setting up multi-dimensional transfer funct…
Photo realism in computer generated imagery is crucially dependent on how well an artist is able to recreate real-world materials in the scene. The workflow for material modeling and editing typically involves manual tweaking of material parameters and uses a standard path tracing engine for visual feedback. A lot of time may be spent in iterative selection and rendering of materials at an appropriate quality. In this work, we propose a convolutional neural network based workflow which quickly generates high-quality ray traced material visualizations on a shaderball. Our novel architecture all…
Computational fluid dynamic simulations often produce large clusters of finite elements with non-trivial, non-convex boundaries and uneven distributions among compute nodes, posing challenges to compositing during interactive volume rendering. Correct, in-place visualization of such clusters becomes difficult because viewing rays straddle domain boundaries across multiple compute nodes. We propose a GPU-based, scalable, memory-efficient direct volume visualization framework suitable for in~situ and post~hoc usage. Our approach reduces memory usage of the unstructured volume elements by leverag…
A central challenge in data visualization is to understand which data samples are required to generate an image of a data set in which the relevant information is encoded. In this work, we make a first step towards answering the question of whether an artificial neural network can predict where to sample the data with higher or lower density, by learning of correspondences between the data, the sampling patterns and the generated images. We introduce a novel neural rendering pipeline, which is trained end-to-end to generate a sparse adaptive sampling structure from a given low-resolution input…
Ultrasound is one of the most frequently used imaging modality in medicine. The high spatial resolution, its interactive nature and non-invasiveness makes it the first choice in many examinations. Image interpretation is one of ultrasound's main challenges. Much training is required to obtain a confident skill level in ultrasound-based diagnostics. State-of-the-art graphics techniques is needed to provide meaningful visualizations of ultrasound in real-time. In this paper we present the process-pipeline for ultrasound visualization, including an overview of the tasks performed in the specific …
We present a framework to interactively volume-render three-dimensional data cubes using distributed ray-casting and volume bricking over a cluster of workstations powered by one or more graphics processing units (GPUs) and a multi-core CPU. The main design target for this framework is to provide an in-core visualization solution able to provide three-dimensional interactive views of terabyte-sized data cubes. We tested the presented framework using a computing cluster comprising 64 nodes with a total of 128 GPUs. The framework proved to be scalable to render a 204 GB data cube with an average…
