We present a two-component Machine Learning (ML) based approach for classifying astronomical images by data-quality via an examination of sources detected in the images and image pixel values from representative sources within those images. The first component, which uses a clustering algorithm, creates a proper and small fraction of the image pixels to determine the quality of the observation. The representative images (and associated tables) are ~800 times smaller than the original images, significantly reducing the time required to train our algorithm. The useful information in the images is preserved, permitting them to be classified in different categories, but the required storage is reduced. The second component, which is a deep neural network model, classifies the representative images. Using ground-based telescope imaging data, we demonstrate that the method can be used to separate 'usable' images from those that present some problems for scientific projects -- such as images that were taken in sub-optimal conditions. This method uses two different data sets as input to a deep model and provides better performance than if we only used the images' pixel information. The method may be used in cases where large and complex data sets should be examined using deep models. Our automated classification approach achieves 97% agreement when compared to classification generated via manual image inspection. We compare our method with traditional results and show that the method improves the results by about 10%, and also presents more comprehensive outcomes.
Ellesmere Island, at the most northerly tip of Canada, possesses the highest mountain peaks within 10 degrees of the pole. The highest is 2616 m, with many summits over 1000 m, high enough to place them above a stable low-elevation thermal inversion that persists through winter darkness. Our group has studied four mountains along the northwestern coast which have the additional benefit of smooth onshore airflow from the ice-locked Arctic Ocean. We deployed small robotic site testing stations at three sites, the highest of which is over 1600 m and within 8 degrees of the pole. Basic weather and…
The internal antarctic plateau revealed in the last years to be a site with interesting potentialities for the astronomical applications due to the extreme dryness and low temperatures, the typical high altitude of the plateau, the weak level of turbulence in the free atmosphere down to a just few tens of meters from the ground and the thin optical turbulence layer developed at the ground. The main goal of a site testing assessment above the internal antarctic plateau is to characterize the site (optical turbulence and classical meteorological parameters) and to quantify which is the gain we m…
Ground-based optical astronomy necessarily involves sensing the light of astronomical objects along with the contributions of many natural sources ranging from the Earth's atmosphere to cosmological light. In addition, astronomers have long contended with artificial light pollution that further adds to the 'background' against which astronomical objects are seen. Understanding the brightness of the night sky is therefore fundamental to astronomy. The last comprehensive review of this subject was nearly a half-century ago, and we have learned much about both the natural and artificial night sky…
We have developed a method that maps large astronomical images onto a two-dimensional map and clusters them. A combination of various state-of-the-art machine learning (ML) algorithms is used to develop a fully unsupervised image quality assessment and clustering system. Our pipeline consists of a data pre-processing step where individual image objects are identified in a large astronomical image and converted to smaller pixel images. This data is then fed to a deep convolutional autoencoder jointly trained with a self-organizing map (SOM). This part can be used as a recommendation system. The…
Modern interferometers routinely provide radio-astronomical images down to subarcsecond resolution. However, interferometers filter out spatial scales larger than those sampled by the shortest baselines, which affects the measurement of both spatial and spectral features. Complementary single-dish data are vital for recovering the true flux distribution of spatially resolved astronomical sources with such extended emission. In this work, we provide an overview of the prominent available methods to combine single-dish and interferometric observations. We test each of these methods in the framew…
We present a classification-based approach to identify quasi-stellar radio sources (quasars) in the Sloan Digital Sky Survey and evaluate its performance on a manually labeled training set. While reasonable results can already be obtained via approaches working only on photometric data, our experiments indicate that simple but problem-specific features extracted from spectroscopic data can significantly improve the classification performance. Since our approach works orthogonal to existing classification schemes used for building the spectroscopic catalogs, our classification results are well …
KMOS is a multi-object near-infrared integral field spectrometer with 24 deployable cryogenic pick-off arms. Inevitably, data processing is a complex task that requires careful calibration and quality control. In this paper we describe all the steps involved in producing science-quality data products from the raw observations. In particular, we focus on the following issues: (i) the calibration scheme which produces maps of the spatial and spectral locations of all illuminated pixels on the detectors; (ii) our concept of minimising the number of interpolations, to the limiting case of a single…
I present a software tool for solving the astrometry of astronomical images. The code puts emphasis on robustness against failures for correctly matching the sources in the image to a reference catalog, and on the stability of the solutions over the field of view (e.g., using orthogonal polynomials for the fitted transformation). The code was tested on over 50,000 images from various sources, including the Palomar Transient Factory (PTF) and the Zwicky Transient Facility (ZTF). The tested images equally represent low and high Galactic latitude fields and exhibit failure/bad-solution rate of <2…
Proper modelling of astronomical receivers is vital: it describes the systematic errors in the raw data, guides the receiver design process, and assists data calibration. In this paper we describe a method of analytically modelling the full signal and noise behaviour of arbitrarily complex radio receivers. We use electrical scattering matrices to describe the signal behaviour of individual components in the receiver, and noise correlation matrices to describe their noise behaviour. These are combined to produce the full receiver model. We apply this approach to a specified receiver architectur…
The rising population of artificial satellites and associated debris in low-altitude orbits is increasing the overall brightness of the night sky, threatening ground-based astronomy as well as a diversity of stakeholders and ecosystems reliant on dark skies. We present calculations of the potentially large rise in global sky brightness from space objects, including qualitative and quantitative assessments of how professional astronomy may be affected. Debris proliferation is of special concern: since all log-decades in debris size contribute approximately the same amount of night sky radiance,…
While fully automated methods for detecting faint moving objects in astronomical images - such as Kuiper belt objects (KBOs) - are constantly improving, visual detection still has a role to play especially when the fixed background is cluttered with stars. Color intensity projections (CIPs) using hue cycling - which combines a sequence of greyscale images into a single color image - aids in the visual detection of moving objects by highlighting them using color in an intuitive way. To demonstrate the usefulness of CIPs in detecting faint moving objects a sequence of 16 images from the SuprimeC…
Operation of the US Virtual Astronomical Observatory shares some issues with modern physical observatories, e.g., intimidating data volumes and rapid technological change, and must also address unique concerns like the lack of direct control of the underlying and scattered data resources, and the distributed nature of the observatory itself. In this paper we discuss how the VAO has addressed these challenges to provide the astronomical community with a coherent set of science-enabling tools and services. The distributed nature of our virtual observatory-with data and personnel spanning geograp…
The volume of data that will be produced by the next generation of astrophysical instruments represents a significant opportunity for making unplanned and unexpected discoveries. Conversely, finding unexpected objects or phenomena within such large volumes of data presents a challenge that may best be solved using computational and statistical approaches. We present the application of a coarse-grained complexity measure for identifying interesting observations in large astronomical data sets. This measure, which has been termed apparent complexity, has been shown to model human intuition and p…
The volume of radio-astronomical data is a considerable burden in the processing and storing of radio observations with high time and frequency resolutions and large bandwidths. Lossy compression of interferometric radio-astronomical data is considered to reduce the volume of visibility data and to speed up processing. A new compression technique named "Dysco" is introduced that consists of two steps: a normalization step, in which grouped visibilities are normalized to have a similar distribution; and a quantization and encoding step, which rounds values to a given quantization scheme using a…
Self-Driving Telescopes: Autonomous Scheduling of Astronomical Observation Campaigns with Offline Reinforcement Learning
University of Pisa（比萨大学）；Fermi National Accelerator Laboratory（费米国家加速器实验室）；Kavli Institute for Cosmological Physics（卡夫利宇宙物理研究所）；University of Chicago（芝加哥大学）；Space Telescope Science Institute（太空望远镜科学研究所）
Carnegie Mellon University（卡内基梅隆大学）；New York University（纽约大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
We describe a system that builds a high dynamic-range and wide-angle image of the night sky by combining a large set of input images. The method makes use of pixel-rank information in the individual input images to improve a "consensus" pixel rank in the combined image. Because it only makes use of ranks and the complexity of the algorithm is linear in the number of images, the method is useful for large sets of uncalibrated images that might have undergone unknown non-linear tone mapping transformations for visualization or aesthetic reasons. We apply the method to images of the night sky (of…
Lucky imaging is a high-resolution astronomical image recovery technique with two classic implementation algorithms, i.e. image selecting, shifting and adding in image space and data selecting and image synthesizing in Fourier space. This paper proposes a novel lucky imaging algorithm where with space-domain and frequency-domain selection rates as a link, the two classic algorithms are combined successfully, making each algorithm a proper subset of the novel hybrid algorithm. Experimental results show that with the same experiment dataset and platform, the high-resolution image obtained by the…
We present an extension of the multi-band galaxy fitting method scarlet which allows the joint modeling of astronomical images from different instruments, by performing simultaneous resampling and convolution. We introduce a fast and formally accurate linear projection operation that maps a pixelated model at a given resolution onto an observation frame with a different point spread function and pixel scale. We test our implementation against the well-tested resampling and convolution method in galsim on simulated images mimicking observations with the Euclid space telescope and the Vera C. Ru…
Large modern surveys require efficient review of data in order to find transient sources such as supernovae, and to distinguish such sources from artefacts and noise. Much effort has been put into the development of automatic algorithms, but surveys still rely on human review of targets. This paper presents an integrated system for the identification of supernovae in data from Pan-STARRS1, combining classifications from volunteers participating in a citizen science project with those from a convolutional neural network. The unique aspect of this work is the deployment, in combination, of both …
Astronomical objects frequently exhibit structure over a wide range of scales whereas many telescopes, especially interferometer arrays, only sample a limited range of spatial scales. In order to properly image these objects, images from a set of instruments covering the range of scales may be needed. These images then must be combined in a manner to recover all spatial scales. This paper describes the feathering technique for image combination in the Fourier transform plane. Implementations in several packages are discussed and example combinations of single dish and interferometric observati…
This paper presents multi-filter measurements of the night sky brightness at the South African Astronomical Observatory (SAAO) in Sutherland in the presence of a bright moon. The observations cover a wide range of sky directions, lunar phases and lunar positions. A revised simplified scattering model is developed for estimating the sky brightness due to moonlight that more accurately reflects the atmospheric extinction of the lunar beam compared to models frequently applied in astronomical studies. Contributions to night sky brightness due to sources other than moonlight are quantified and sub…
Large Language Models for Limited Noisy Data: A Gravitational Wave Identification Study
Institute of Fundamental Physics and Quantum Technology（基础物理与量子技术研究所）；School of Physical Science and Technology（物理科学与技术学院）；School of Mathematics and Physics（数学与物理学院）
