We demonstrate optics-free imaging of complex QR-codes using a bare image sensor and a trained artificial neural network (ANN). The ANN is trained to interpret the raw sensor data for human visualization. The image sensor is placed at a specified gap from the QR code. We studied the robustness of our approach by experimentally testing the output of the ANNs with system perturbations of this gap, and the translational and rotational alignments of the QR code to the image sensor. Our demonstration opens us the possibility of using completely optics-free cameras for application-specific imaging of complex, non-sparse objects.
A growing number of applications require the reconstructionof 3D objects from a very small number of views. In this research, we consider the problem of reconstructing a 3D object from only 4 Flash X-ray CT views taken during the impact of a Kolsky bar. For such ultra-sparse view datasets, even model-based iterative reconstruction (MBIR) methods produce poor quality results. In this paper, we present a framework based on a generalization of Plug-and-Play, known as Multi-Agent Consensus Equilibrium (MACE), for incorporating complex and nonlinear prior information into ultra-sparse CT reconstruc…
Increasing the resolution of image sensors has been a never ending struggle since many years. In this paper, we propose a novel image sensor layout which allows for the acquisition of images at a higher resolution and improved quality. For this, the image sensor makes use of non-regular sampling which reduces the impact of aliasing. Therewith, it allows for capturing details which would not be possible with state-of-the-art sensors of the same number of pixels. The non-regular sampling is achieved by rotating prototype pixel cells in a non-regular fashion. As not the whole area of the pixel ce…
It has previously been demonstrated that model-based reconstruction methods relying on a priori knowledge of the imaging point spread function (PSF) coupled to sparsity priors on the object to image can provide super-resolution in photoacoustic (PA) or in ultrasound (US) imaging. Here, we experimentally show that such reconstruction also leads to super-resolution in both PA and US imaging with arrays having much less elements than used conventionally (sparse arrays). As a proof of concept, we obtained super-resolution PA and US cross-sectional images of microfluidic channels with only 8 elemen…
Noninvasive reconstruction of cardiac electrical activity from surface electrocardiograms (ECG) involves solving an ill-posed inverse problem. Cardiac electrophysiological (EP) models have been used as important a priori knowledge to constrain this inverse problem. However, the reconstruction suffer from inaccuracy and uncertainty of the prior model itself which could be mitigated by estimating a priori model error. Unfortunately, due to the need to handle an additional large number of unknowns in a problem that already suffers from ill-posedness, model error estimation remains an unresolved c…
Deep learning affords enormous opportunities to augment the armamentarium of biomedical imaging, albeit its design and implementation have potential flaws. Fundamentally, most deep learning models are driven entirely by data without consideration of any prior knowledge, which dramatically increases the complexity of neural networks and limits the application scope and model generalizability. Here we establish a geometry-informed deep learning framework for ultra-sparse 3D tomographic image reconstruction. We introduce a novel mechanism for integrating geometric priors of the imaging system. We…
Universidad de Zaragoza（萨拉戈萨大学）；Stanford University（斯坦福大学）；MPI Informatik（马克斯·普朗克信息学研究所）
Current HDR acquisition techniques are based on either (i) fusing multibracketed, low dynamic range (LDR) images, (ii) modifying existing hardware and capturing different exposures simultaneously with multiple sensors, or (iii) reconstructing a single image with spatially-varying pixel exposures. In this paper, we propose a novel algorithm to recover high-quality HDRI images from a single, coded exposure. The proposed reconstruction method builds on recently-introduced ideas of convolutional sparse coding (CSC); this paper demonstrates how to make CSC practical for HDR imaging. We demonstrate …
Tongji University（同济大学）；Universidad Industrial de Santander（桑坦德工业大学）；Kunming University of Science and Technology（昆明理工大学）；University of Delaware（特拉华大学）
This paper aims at developing a clustering approach with spectral images directly from CASSI compressive measurements. The proposed clustering method first assumes that compressed measurements lie in the union of multiple low-dimensional subspaces. Therefore, sparse subspace clustering (SSC) is an unsupervised method that assigns compressed measurements to their respective subspaces. In addition, a 3D spatial regularizer is added into the SSC problem, thus taking full advantages of the spatial information contained in spectral images. The performance of the proposed spectral image clustering a…
University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University（上海交通大学密西根学院）；Peking University（北京大学）
Photoacoustic microscopy (PAM) has been a promising biomedical imaging technology in recent years. However, the point-by-point scanning mechanism results in low-speed imaging, which limits the application of PAM. Reducing sampling density can naturally shorten image acquisition time, which is at the cost of image quality. In this work, we propose a method using convolutional neural networks (CNNs) to improve the quality of sparse PAM images, thereby speeding up image acquisition while keeping good image quality. The CNN model utilizes both squeeze-and-excitation blocks and residual blocks to a…
MALDI mass spectrometry imaging (MALDI MSI) is a spatially resolved analytical tool for biological tissue analysis by measuring mass-to-charge ratios of ionized molecules. With increasing spatial and mass resolution of MALDI MSI data, appropriate data analysis and interpretation is getting more and more challenging. A reliable separation of important peaks from noise (aka peak detection) is a prerequisite for many subsequent processing steps and should be as accurate as possible. We propose a novel peak detection algorithm based on sparse frame multipliers, which can be applied to raw MALDI MS…
SparseSSP: 3D Subcellular Structure Prediction from Sparse-View Transmitted Light Images
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Jinan University（暨南大学）
Super-temporal-resolution Photoacoustic Imaging with Dynamic Reconstruction through Implicit Neural Representation in Sparse-view
School of Information Science and Technology, ShanghaiTech University（信息科学与技术学院，上海科技大学）；School of Biomedical Engineering, ShanghaiTech University（生物医学工程学院，上海科技大学）；School of Biomedical Engineering and Institute of Medical Robotics, Shanghai Jiao Tong University（生物医学工程学院与医学机器人研究所，上海交通大学）；School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China（生物医学工程学院，生命科学与医学系，中国科学技术大学）；School of Engineering Science, University of Science and Technology of China（工程科学学院，中国科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（苏州先进研究所，中国科学技术大学）
ShanghaiTech University（上海科技大学）；Chinese Academy of Sciences（中国科学院）；Shanghai Institute of Microsystem and Information Technology（上海微系统与信息技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Photoacoustic (PA) imaging is a biomedical imaging modality capable of acquiring high-contrast images of optical absorption at depths much greater than traditional optical imaging techniques. However, practical instrumentation and geometry limit the number of available acoustic sensors surrounding the imaging target, which results in the sparsity of sensor data. Conventional PA image reconstruction methods give severe artifacts when they are applied directly to the sparse PA data. In this paper, we firstly propose to employ a novel signal processing method to make sparse PA raw data more suita…
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）
In dynamic magnetic resonance (MR) imaging, low-rank plus sparse (L+S) decomposition, or robust principal component analysis (PCA), has achieved stunning performance. However, the selection of the parameters of L+S is empirical, and the acceleration rate is limited, which are common failings of iterative compressed sensing MR imaging (CS-MRI) reconstruction methods. Many deep learning approaches have been proposed to address these issues, but few of them use a low-rank prior. In this paper, a model-based low-rank plus sparse network, dubbed L+S-Net, is proposed for dynamic MR reconstruction. I…
Magnetic Particle Imaging (MPI) is a preclinical imaging technique capable of visualizing the spatio-temporal distribution of magnetic nanoparticles. The image reconstruction of this fast and dynamic process relies on efficiently solving an ill-posed inverse problem. Current approaches to reconstruct the tracer concentration from its measurements are either adapted to image characteristics of MPI but suffer from higher computational complexity and slower convergence or are fast but lack in the image quality of the reconstructed images. In this work we propose a novel MPI reconstruction method …
Optical imaging through scattering media is a commonly confronted with the problem of reconstruction of complex objects and optical memory effect. To solve the problem, here, we propose a novel configuration based on the combination of ptychography and shower-curtain effect, which enables the retrieval of non-sparse samples through scattering media beyond the memory effect. Furthermore, by virtue of the shower-curtain effect, the proposed imaging system is insensitive to dynamic scattering media. Results from the retrieval of hair follicle section demonstrate the effectiveness and feasibility …
IMT Atlantique（大西洋国立高等矿业电信学校）；LaTIM UMR 1101, Inserm（法国国家健康与医学研究院LaTIM UMR 1101实验室）；Centre Hospitalier Régional et Universitaire (CHRU) de Brest（布雷斯特大学区域中心医院）；Fondation ILDYS（ILDYS基金会）；Symbiosis Center for Medical Image Analysis, Symbiosis International University（共生国际大学共生医学图像分析中心）
Clinical diagnosis of the pediatric musculoskeletal system relies on the analysis of medical imaging examinations. In the medical image processing pipeline, semantic segmentation using deep learning algorithms enables an automatic generation of patient-specific three-dimensional anatomical models which are crucial for morphological evaluation. However, the scarcity of pediatric imaging resources may result in reduced accuracy and generalization performance of individual deep segmentation models. In this study, we propose to design a novel multi-task, multi-domain learning framework in which a …
Dynamic Contrast-enhanced magnetic resonance imaging (DCE-MRI) is a tissue perfusion imaging technique. Some versatile free-breathing DCE-MRI techniques combining compressed sensing (CS) and parallel imaging with golden-angle radial sampling have been developed to improve motion robustness with high spatial and temporal resolution. These methods have demonstrated good diagnostic performance in clinical setting, but the reconstruction quality will degrade at high acceleration rates and overall reconstruction time remains long. In this paper, we proposed a new parallel CS reconstruction model fo…
Shenzhen University（深圳大学）；Nanophotonics Research Center（纳米光子学研究中心）
Deep learning has been extensively applied in many optical imaging applications in recent years. Despite the success, the limitations and drawbacks of deep learning in optical imaging have been seldom investigated. In this work, we show that conventional linear-regression-based methods can outperform the previously proposed deep learning approaches for two black-box optical imaging problems in some extent. Deep learning demonstrates its weakness especially when the number of training samples is small. The advantages and disadvantages of linear-regression-based methods and deep learning are ana…
Inverse Problem Approach to Aberration Correction for in vivo Transcranial Imaging Based on a Sparse Representation of Contrast-enhanced Ultrasound Data
Atmospheric turbulence is a serious problem for traditional optical imaging, especially for satellite and aircraft-to-ground imaging. Here, we report a novel and practical phenomenon in which turbulence-free images can be reconstructed on a conventional optical camera based on the accumulation of sunlight intensity and photon number fluctuation autocorrelation. Different from conventional ghost imaging, this method can obtain turbulence-free images, and its imaging speed is comparable to that of traditional optical imaging. Moreover, by adding photon number fluctuation autocorrelation algorith…
Many imaging technologies rely on tomographic reconstruction, which requires solving a multidimensional inverse problem given a finite number of projections. Backprojection is a popular class of algorithm for tomographic reconstruction, however it typically results in poor image reconstructions when the projection angles are sparse and/or if the sensors characteristics are not uniform. Several deep learning based algorithms have been developed to solve this inverse problem and reconstruct the image using a limited number of projections. However these algorithms typically require examples of th…
In single-molecule super-resolution microscopy, engineered point-spread functions (PSFs) are designed to efficiently encode new molecular properties, such as 3D orientation, into complex spatial features captured by a camera. To fully benefit from their optimality, algorithms must estimate multi-dimensional parameters such as molecular position and orientation in the presence of PSF overlap and model-experiment mismatches. Here, we present a novel joint sparse deconvolution algorithm based on the decomposition of fluorescence images into six basis images that characterize molecular orientation…
University of Cambridge（剑桥大学）；University of Cincinnati（辛辛那提大学）
Quantitative cancer image analysis relies on the accurate delineation of tumours, a very specialised and time-consuming task. For this reason, methods for automated segmentation of tumours in medical imaging have been extensively developed in recent years, being Computed Tomography one of the most popular imaging modalities explored. However, the large amount of 3D voxels in a typical scan is prohibitive for the entire volume to be analysed at once in conventional hardware. To overcome this issue, the processes of downsampling and/or resampling are generally implemented when using traditional …
Visual object recognition is one of the most important perception functions for a wide range of intelligent machines. A conventional recognition process begins with forming a clear optical image of the object, followed by its computer analysis. In contrast, it is possible to carry out recognition without imaging by using coherent illumination and directly analyzing the optical interference pattern of the scattered light as captured by an image sensor. Here we show that such direct visual recognition can overcome traditional limitations of imaging optics to realize excellent recognition without…
Eindhoven University of Technology（埃因霍温理工大学）；University of Amsterdam（阿姆斯特丹大学）；Philips Research（飞利浦研究院）
Limitations on bandwidth and power consumption impose strict bounds on data rates of diagnostic imaging systems. Consequently, the design of suitable (i.e. task- and data-aware) compression and reconstruction techniques has attracted considerable attention in recent years. Compressed sensing emerged as a popular framework for sparse signal reconstruction from a small set of compressed measurements. However, typical compressed sensing designs measure a (non)linearly weighted combination of all input signal elements, which poses practical challenges. These designs are also not necessarily task-o…
Argonne National Laboratory（阿贡国家实验室）；Stats Perform；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
Ptychography has rapidly grown in the fields of X-ray and electron imaging for its unprecedented ability to achieve nano or atomic scale resolution while simultaneously retrieving chemical or magnetic information from a sample. A ptychographic reconstruction is achieved by means of solving a complex inverse problem that imposes constraints both on the acquisition and on the analysis of the data, which typically precludes real-time imaging due to computational cost involved in solving this inverse problem. In this work we propose PtychoNN, a novel approach to solve the ptychography reconstructi…
Three-dimensional (3D) X-ray imaging techniques like tomography and confocal microscopy are crucial for academic and industrial applications. These approaches access 3D information by scanning the sample with respect to the X-ray source. However, the scanning process limits the temporal resolution when studying dynamics and is not feasible for some applications, such as surgical guidance in medical applications. Alternatives to obtaining 3D information when scanning is not possible are X-ray stereoscopy and multi-projection imaging. However, these approaches suffer from limited volumetric info…
Optical tomographic cross-sectional images of biological samples were made possible by interferometric imaging techniques such as Optical Coherence Tomography (OCT). Owing to its unprecedented view of the sample, OCT has become a gold standard, namely for human retinal imaging in the clinical environment. In this Letter, we present Optical Incoherence Tomography (OIT): a completely digital method extending the possibility to generate tomographic retinal cross-sections to non-interferometric imaging systems such as en-face AO-ophthalmoscopes. We demonstrate that OIT can be applied to different …
Novel data acquisition schemes have been an emerging need for scanning microscopy based imaging techniques to reduce the time in data acquisition and to minimize probing radiation in sample exposure. Varies sparse sampling schemes have been studied and are ideally suited for such applications where the images can be reconstructed from a sparse set of measurements. Dynamic sparse sampling methods, particularly supervised learning based iterative sampling algorithms, have shown promising results for sampling pixel locations on the edges or boundaries during imaging. However, dynamic sampling for…
