Hadamard single-pixel imaging (HSI) is an appealing imaging technique due to its features of low hardware complexity and industrial cost. To improve imaging efficiency, many studies have focused on sorting Hadamard patterns to obtain reliable reconstructed images with very few samples. In this study, we present an efficient HSI imaging method that employs an exponential probability function to sample Hadamard spectra along a direction with better energy concentration for obtaining Hadamard patterns. We also propose an XY order to further optimize the pattern-selection method with extremely fast Hadamard order generation while retaining the original performance. We used the compressed sensing algorithm for image reconstruction. The simulation and experimental results show that these pattern-selection method reliably reconstructs objects and preserves the edge and details of images.
Single-pixel imaging via compressed sensing can reconstruct high-quality images from a few linear random measurements of an object/scene known a priori to be sparse or compressive, by using a point/bucket detector without spatial resolution. Nevertheless, it still faces a harsh trade-off among the acquisition time, the spatial resolution and the signal-to-noise ratio. Here we present a new compressive imaging approach with use of a strategy called cake-cutting which optimally reorders the deterministic Hadamard basis. By this means, the number of measurements can be dramatically reduced by mor…
Single-pixel imaging (SPI) is very popular in subsampling applications, but the random measurement matrices it typically uses will lead to measurement blindness as well as difficulties in calculation and storage, and will also limit the further reduction in sampling rate. The deterministic Hadamard basis has become an alternative choice due to its orthogonality and structural characteristics. There is evidence that sorting the Hadamard basis is beneficial to further reduce the sampling rate, thus many orderings have emerged, but their relations remain unclear and lack a unified theory. Given t…
One of the most active research fields in single-pixel imaging is the influence of the sampling basis and its order in the quality of the reconstructed images. This paper presents two new orders, ascending scale (AS) and ascending inertia (AI), of the Hadamard basis and test their performance, using simulation and experimental methods, for low sampling ratios (0.5 to 0.01). These orders were compared with two state-of-the-art orders, cake-cutting (CC) and total gradient (TG), using TVAL3 as the reconstruction algorithm and three noise levels. These newly proposed orders have better reconstruct…
In previous single-pixel imaging systems, the light source was generally idle with respect to time. Here, we propose a novel image fusion and visible watermarking scheme based on Fourier single-pixel imaging (FSPI) with a multiplexed time-varying (TV) signal, which is generated by the watermark pattern hidden in the light source. We call this scheme as TV-FSPI. With TV-FSPI, we can realize high-quality visible image watermarking, encrypted image watermarking and full-color visible image watermarking. We also discuss the extension to invisible watermarking based on TV-FSPI. Furthermore, we don'…
Jinan University（暨南大学）；Guangdong Communication Polytechnic（广东通信职业技术学院）
Fourier single-pixel imaging (FSI) is a branch of single-pixel imaging techniques. It uses Fourier basis patterns as structured patterns for spatial information acquisition in the Fourier domain. However, the spatial resolution of the image reconstructed by FSI mainly depends on the number of Fourier coefficients sampled. The reconstruction of a high-resolution image typically requires a number of Fourier coefficients to be sampled, and therefore takes a long data acquisition time. Here we propose a new sampling strategy for FSI. It allows FSI to reconstruct a clear and sharp image with a redu…
When imaging moving objects, single-pixel imaging produces motion blur. This paper proposes a new single-pixel imaging method, which can achieve anti-motion blur imaging of a fast-moving object. The geometric moment patterns and Hadamard patterns are used to alternately encode the position information and the image information of the object with time-division multiplexing. In the reconstruction process, the object position information is extracted independently and combining motion-compensation reconstruction algorithm to decouple the object motion from image information. As a result, the anti…
Image-free tracking methods based on single-pixel detectors (SPDs) can track a moving object at a very high frame rate, but they rarely can achieve simultaneous imaging of such an object. In this study, we propose a method for simultaneously obtaining the relative displacements and images of a translational object. Four binary Fourier patterns and two differential Hadamard patterns are used to modulate one frame of the object and then modulated light signals are obtained by SPD. The relative displacements and image of the moving object can be gradually obtained along with the detection. The pr…
We propose a method of reduction of experimental noise in single-pixel imaging by expressing the subsets of sampling patterns as linear combinations of vertices of a multidimensional regular simplex. This method may be also directly extended to complementary sampling. The modified sampling consists only of non-negative patterns. The measurement becomes theoretically independent of the ambient illumination, and in practice becomes more robust to the varying conditions of the experiment. We show how the optimal dimensionality of the simplex depends on the level of measurement noise. We present e…
Single-pixel compressive imaging can recover images from a small amount of measurements, offering many benefits especially for the scenes where the array detection is unavailable. However, the widely used random patterns fail to explore internal relations between the patterns and the image reconstruction. Here we propose a single-pixel imaging method based on origami pattern construction with a better imaging quality, but with less uncertainty of the pattern sequence. It can decrease the sampling ratio even to 0.5\%, really realizing super sub-Nyquist sampling. The experimental realization of …
A hybrid classical-quantum approach for evaluation of multi-dimensional Walsh-Hadamard transforms and its applications to quantum image processing are proposed. In this approach, multidimensional Walsh-Hadamard transforms are obtained using quantum Hadamard gates (along with state-preparation, shifting, scaling and measurement operations). The proposed approach for evaluation of multidimensional Walsh-Hadamard transform has a considerably lower computational complexity (involving $O(N^d)$ operations) in contrast to classical Fast Walsh-Hadamard transform (involving $O(N^d~\log_2 N^d)$ operatio…
Computational ghost imaging or single-pixel imaging enables the image formation of an unknown scene using a lens-free photodetector. In this Letter, we present a computational panoramic ghost imaging system that can achieve the full-color panorama using a single-pixel photodetector, where a convex mirror performs the optical transformation of the engineered Hadamard-based circular illumination pattern from unidirectionally to omnidirectionally. To our best knowledge, it is the first time to propose the concept of ghost panorama and realize preliminary experimentations. It is foreseeable that g…
Single-pixel imaging (SPI) has a major drawback that many sequential illuminations are required for capturing one single image with long acquisition time. Basis illumination patterns such as Fourier patterns and Hadamard patterns can achieve much better imaging efficiency than random patterns. But the performance is still sub-optimal since the basis patterns are fixed and non-adaptive for varying object images. This Letter proposes a novel scheme for designing and optimizing the illumination patterns adaptively from an image dictionary by extracting the common image features using principal co…
Under weak illumination, tracking and imaging moving object turns out to be hard. By spatially collecting the signal, single pixel imaging schemes promise the capability of image reconstruction from low photon flux. However, due to the requirement on large number of samplings, how to clearly image moving objects is an essential problem for such schemes. Here we present a principle of single pixel tracking and imaging method. Velocity vector of the object is obtained from temporal correlation of the bucket signals in a typical computational ghost imaging system. Then the illumination beam is st…
Computer generated holography (CGH) has seen a resurgence in recent years due, in part, to the rise of virtual and mixed reality systems. The majority of approaches for CGH are based on a sampled Discrete Fourier Transform (DFT) and ignore the interstitial behaviour between sampling points in the replay field. In this paper we demonstrate that neighbouring replay field pixels can interfere significantly giving the visual impression of pixel movement and increased noise. We also demonstrate that increasing the separation between target pixels reduces the interference and improves pixel quality.…
School of Information and Electronics & Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology（北京理工大学信息与电子学院及多学科科学前沿研究研究院）
Using single-pixel detection, the end-to-end neural network that jointly optimizes both encoding and decoding enables high-precision imaging and high-level semantic sensing. However, for varied sampling rates, the large-scale network requires retraining that is laboursome and computation-consuming. In this letter, we report a weighted optimization technique for dynamic rate-adaptive single-pixel imaging and sensing, which only needs to train the network for one time that is available for any sampling rates. Specifically, we introduce a novel weighting scheme in the encoding process to characte…
In this letter we report on the effects of pixel crosstalk on the experimental realization of a reported encoding method (Opt. Lett. 39, 1740 (2014)) with PA-LCoS SLMs. We found that, under Nyquist limit condition, about 70% of a single pixel cell can generate unexpected phase modulation. In order to approach uniform phase modulation, and consequently improve the quality of measured amplitude and phase images, a generalized sampling scheme is proposed. To corroborate our proposal, proper experiments were carried out. On this point, a particular implementation of the well-established phase shif…
Structured-light three-dimensional (3D) imaging can achieve 3D shape of a stationary object via one or more pixelated array cameras with phase-shifting illumination. In order to extend 3D imaging to moving scenarios, we propose a 3D imaging method with double projection of a single-frame modulated light pattern and a sampling pattern. It can continuously image the moving 3D scene by making multi-pixel detector axial flat brush scan along the motion axis. Utilizing spatial multiplexing for multiple single-pixel imaging, each single-pixel does not need to keep staring at some part of the object,…
Increasing the resolution of image sensors has been a never ending struggle since many years. In this paper, we propose a novel image sensor layout which allows for the acquisition of images at a higher resolution and improved quality. For this, the image sensor makes use of non-regular sampling which reduces the impact of aliasing. Therewith, it allows for capturing details which would not be possible with state-of-the-art sensors of the same number of pixels. The non-regular sampling is achieved by rotating prototype pixel cells in a non-regular fashion. As not the whole area of the pixel ce…
X-ray "ghost" imaging has drawn great attention for its potential to lower radiation dose in medical diagnosis. For practical implementation, however, the efficiency and image quality have to be greatly improved. Here we demonstrate a computational ghost imaging scheme where a bucket detector and specially designed modulation masks are used, together with a new robust deep learning algorithm in which a compressed set of Hadamard matrices is incorporated into a multi-level wavelet convolutional neural network. By this means we have obtained an image of a real object from only 18.75% of the Nyqu…
Single-pixel imaging is an indirect imaging technique which utilizes simplified optical hardware and advanced computational methods. It offers novel solutions for hyper-spectral imaging, polarimetric imaging, three-dimensional imaging, holographic imaging, optical encryption and imaging through scattering media. The main limitations for its use come from relatively high measurement and reconstruction times. In this paper we propose to reduce the required signal acquisition time by using a novel sampling scheme based on a random selection of Morlet wavelets convolved with white noise. While suc…
We present a closed-form image reconstruction method for single pixel imaging based on the generalized inverse of the measurement matrix. Its numerical cost scales linearly with the number of measured samples. Regularization is obtained by minimizing the norms of the convolution between the reconstructed image and a set of spatial filters, and the final reconstruction formula can be expressed in terms of matrix pseudoinverse. At high compression this approach is an interesting alternative to the methods of compressive sensing based on l1-norm optimization, which are too slow for real-time appl…
In this paper, we propose an automatic brain tumor segmentation approach (e.g., PixelNet) using a pixel-level convolutional neural network (CNN). The model extracts feature from multiple convolutional layers and concatenate them to form a hyper-column where samples a modest number of pixels for optimization. Hyper-column ensures both local and global contextual information for pixel-wise predictors. The model confirms the statistical efficiency by sampling a few pixels in the training phase where spatial redundancy limits the information learning among the neighboring pixels in conventional pi…
Quarter sampling is a novel sensor concept that enables the acquisition of higher resolution images without increasing the number of pixels. This is achieved by covering three quarters of each pixel of a low-resolution sensor such that only one quadrant of the sensor area of each pixel is sensitive to light. By randomly masking different parts, effectively a non-regular sampling of a higher resolution image is performed. Combining a properly designed mask and a high-quality reconstruction algorithm, a higher image quality can be achieved than using a low-resolution sensor and subsequent upsamp…
The usually reported pixel resolution of single pixel imaging (SPI) varies between $32 \times 32$ and $256 \times 256$ pixels falling far below imaging standards with classical methods. Low resolution results from the trade-off between the acceptable compression ratio, the limited DMD modulation frequency, and reasonable reconstruction time, and has not improved significantly during the decade of intensive research on SPI. In this paper we show that image measurement at the full resolution of the DMD, which lasts only a fraction of a second, is possible for sparse images or in a situation when…
Non-regular sampling can reduce aliasing at the expense of noise. Recently, it has been shown that non-regular sampling can be carried out using a conventional regular imaging sensor when the surface of its individual pixels is partially covered. This technique is called quarter sampling (also 1/4 sampling), since only one quarter of each pixel is sensitive to light. For this purpose, the choice of a proper sampling mask is crucial to achieve a high reconstruction quality. In the scope of this work, we present an iterative algorithm to improve an arbitrary quarter sampling mask which results i…
We present a compact, diffuser-assisted, single-pixel computational camera. A rotating ground glass diffuser is adopted, in preference to a commonly used digital micro-mirror device (DMD), to encode a two-dimensional (2D) image into single-pixel signals. We retrieve images with an 8.8% sampling ratio after the calibration of the pseudo-random pattern of the diffuser under incoherent illumination. Furthermore, we demonstrate hyperspectral imaging with line array detection by adding a diffraction grating. The implementation results in a cost-effective single-pixel camera for high-dimensional ima…
Advances in CMOS technology have made high resolution image sensors possible. These image sensor pose significant challenges in terms of the amount of raw data generated, energy efficiency and frame rate. This paper presents a new design methodology for an imaging system and a simplified novel image sensor pixel design to be used in such system so that Compressed Sensing (CS) technique can be implemented easily at the sensor level. This results in significant energy savings as it not only cuts the raw data rate but also reduces transistor count per pixel, decreases pixel size, increases fill f…
Technion - Israel Institute of Technology（以色列理工学院）
This work considers the problem of depth completion, with or without image data, where an algorithm may measure the depth of a prescribed limited number of pixels. The algorithmic challenge is to choose pixel positions strategically and dynamically to maximally reduce overall depth estimation error. This setting is realized in daytime or nighttime depth completion for autonomous vehicles with a programmable LiDAR. Our method uses an ensemble of predictors to define a sampling probability over pixels. This probability is proportional to the variance of the predictions of ensemble members, thus …
Recently, non-regular three-quarter sampling has shown to deliver an increased image quality of image sensors by using differently oriented L-shaped pixels compared to the same number of square pixels. A three-quarter sampling sensor can be understood as a conventional low-resolution sensor where one quadrant of each square pixel is opaque. Subsequent to the measurement, the data can be reconstructed on a regular grid with twice the resolution in both spatial dimensions using an appropriate reconstruction algorithm. For this reconstruction, local joint sparse deconvolution and extrapolation (L…
Single pixel imaging can reconstruct two-dimensional images of a scene with only a single-pixel detector. It has been widely used for imaging in non-visible bandwidth (e.g., near-infrared and X-ray) where focal-plane array sensors are challenging to be manufactured. In this paper, we propose a generative adversarial network based reconstruction algorithm for single pixel imaging, which demonstrates efficient reconstruction in 10ms and higher quality. We verify the proposed method with both synthetic and real-world experiments, and demonstrate a good quality of reconstruction of a real-world pl…
We demonstrated a CMOS imaging system that adapts each pixel's exposure and sampling rate to capture high dynamic range (HDR) videos. The system consist of a custom designed image sensor with pixel-wise exposure configurability and a real-time pixel exposure controller. These parts operate in a closed-loop to sample, detect and optimize each pixel's exposure and sampling rate to minimize local region's underexposure, overexposure and motion blurring. Exposure control is implemented using all-integrated electronics without external optical modulation. This reduces overall system size and power …
The speed and quality of single-pixel imaging (SPI) are fundamentally limited by image modulation frequency and by the levels of optical noise and compression noise. In an approach to come close to these limits, we introduce a SPI technique, which is inherently differential, and comprises a novel way of measuring the zeroth spatial frequency of images and makes use of varied thresholding of sampling patterns. With the proposed sampling, the entropy of the detection signal is increased in comparison to standard SPI protocols. Image reconstruction is obtained with a single matrix-vector product …
Technion - Israel Institute of Technology（以色列理工学院）
Depth acquisition, based on active illumination, is essential for autonomous and robotic navigation. LiDARs (Light Detection And Ranging) with mechanical, fixed, sampling templates are commonly used in today's autonomous vehicles. An emerging technology, based on solid-state depth sensors, with no mechanical parts, allows fast, adaptive, programmable scans. In this paper, we investigate the topic of adaptive, image-driven, sampling and reconstruction strategies. First, we formulate a piece-wise linear depth model with several tolerance parameters and estimate its validity for indoor and outdoo…
Council for Scientific and Industrial Research of South Africa（南非科学与工业研究委员会）；University of Johannesburg, South Africa（约翰内斯堡大学）
In this paper, we improve image reconstruction in a single-pixel scanning system by selecting an detector optimal field of view. Image reconstruction is based on compressed sensing and image quality is compared to interpolated staring arrays. The image quality comparisons use a "dead leaves" data set, Bayesian estimation and the Peak-Signal-to-Noise Ratio (PSNR) measure. Compressed sensing is explored as an interpolation algorithm and shows with high probability an improved performance compared to Lanczos interpolation. Furthermore, multi-level sampling in a single-pixel scanning system is sim…
Highly Efficient No-reference 4K Video Quality Assessment with Full-Pixel Covering Sampling and Training Strategy
South China University of Technology（华南理工大学）；Alibaba Digital Media & Entertainment Group（阿里巴巴数字媒体与娱乐集团）；Pazhou Lab（琶洲实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Quarter sampling is a novel sensor design that allows for an acquisition of higher resolution images without increasing the number of pixels. When being used for video data, one out of four pixels is measured in each frame. Effectively, this leads to a non-regular spatio-temporal sub-sampling. Compared to purely spatial or temporal sub-sampling, this allows for an increased reconstruction quality, as aliasing artifacts can be reduced. For the fast reconstruction of such sensor data with a fixed mask, recursive variant of frequency selective reconstruction (FSR) was proposed. Here, pixels measu…
Visualizing the details of different cellular structures is of great importance to elucidate cellular functions. However, it is challenging to obtain high quality images of different structures directly due to complex cellular environments. Fluorescence staining is a popular technique to label different structures but has several drawbacks. In particular, label staining is time consuming and may affect cell morphology, and simultaneous labels are inherently limited. This raises the need of building computational models to learn relationships between unlabeled microscopy images and labeled fluo…
Autoregressive models are a class of exact inference approaches with highly flexible functional forms, yielding state-of-the-art density estimates for natural images. Yet, the sequential ordering on the dimensions makes these models computationally expensive and limits their applicability to low-resolution imagery. In this work, we propose Pixel-Pyramids, a block-autoregressive approach employing a lossless pyramid decomposition with scale-specific representations to encode the joint distribution of image pixels. Crucially, it affords a sparser dependency structure compared to fully autoregres…
Colorado State University（科罗拉多州立大学）；Physical Sciences Inc.（物理科学公司）
Sampling is a fundamental aspect of any implementation of compressive sensing. Typically, the choice of sampling method is guided by the reconstruction basis. However, this approach can be problematic with respect to certain hardware constraints and is not responsive to domain-specific context. We propose a method for defining an order for a sampling basis that is optimal with respect to capturing variance in data, thus allowing for meaningful sensing at any desired level of compression. We focus on the Walsh-Hadamard sampling basis for its relevance to hardware constraints, but our approach a…
The existing segmentation techniques require high-fidelity images as input to perform semantic segmentation. Since the segmentation results contain most of edge information that is much less than the acquired images, the throughput gap leads to both hardware and software waste. In this letter, we report an image-free single-pixel segmentation technique. The technique combines structured illumination and single-pixel detection together, to efficiently samples and multiplexes scene's segmentation information into compressed one-dimensional measurements. The illumination patterns are optimized to…
With the proliferation of inexpensive CMOS cameras, medical imaging experiences a noticeable influx of new technologies. While anatomical imaging is based on well-established principles of photography, physiological optical imaging is a relatively novel range of technologies that requires considering a new set of technical and physiological aspects. We discuss several factors (binning, spatial frequency sampling, and distance to the target area) indispensable to getting quantifiable and reproducible results, which are the essence of physiological imaging. We also discussed their implications f…
We introduce a novel video-rate hyperspectral imager with high spatial, and temporal resolutions. Our key hypothesis is that spectral profiles of pixels in a super-pixel of an oversegmented image tend to be very similar. Hence, a scene-adaptive spatial sampling of an hyperspectral scene, guided by its super-pixel segmented image, is capable of obtaining high-quality reconstructions. To achieve this, we acquire an RGB image of the scene, compute its super-pixels, from which we generate a spatial mask of locations where we measure high-resolution spectrum. The hyperspectral image is subsequently…
