We present a new framework to detect various types of variable objects within massive astronomical time-series data. Assuming that the dominant population of objects is non-variable, we find outliers from this population by using a non-parametric Bayesian clustering algorithm based on an infinite GaussianMixtureModel (GMM) and the Dirichlet Process. The algorithm extracts information from a given dataset, which is described by six variability indices. The GMM uses those variability indices to recover clusters that are described by six-dimensional multivariate Gaussian distributions, allowing our approach to consider the sampling pattern of time-series data, systematic biases, the number of data points for each light curve, and photometric quality. Using the Northern Sky Variability Survey data, we test our approach and prove that the infinite GMM is useful at detecting variable objects, while providing statistical inference estimation that suppresses false detection. The proposed approach will be effective in the exploration of future surveys such as GAIA, Pan-Starrs, and LSST, which will produce massive time-series data.
We present variability analysis of data from the Northern Sky Variability Survey (NSVS). Using the clustering method which defines variable candidates as outliers from large clusters, we cluster 16,189,040 light curves, having data points at more than 15 epochs, as variable and non-variable candidates in 638 NSVS fields. Variable candidates are selected depending on how strongly they are separated from the largest cluster and how rarely they are grouped together in eight dimensional space spanned by variability indices. All NSVS light curves are also cross-correlated to the Infrared Astronomic…
We present an algorithm capable of detecting diffuse, dim sources of any size in an astronomical image. These sources often defeat traditional methods for source finding, which expand regions around points of high intensity. Extended sources often have no bright points and are only detectable when viewed as a whole, so a more sophisticated approach is required. Our algorithm operates at all scales simultaneously by considering a tree of nested candidate bounding boxes, and inverts a hierarchical Bayesian generative model to obtain the probability of sources existing at given locations and size…
The multiscale entropy assesses the complexity of a signal across different timescales. It originates from the biomedical domain and was recently successfully used to characterize light curves as part of a supervised machine learning framework to classify stellar variability. We explore the behavior of the multiscale entropy in detail by studying its algorithmic properties in a stellar variability context and by linking it with traditional astronomical time series analysis methods. We subsequently use the multiscale entropy as the basis for an interpretable clustering framework that can distin…
We present a classification-based approach to identify quasi-stellar radio sources (quasars) in the Sloan Digital Sky Survey and evaluate its performance on a manually labeled training set. While reasonable results can already be obtained via approaches working only on photometric data, our experiments indicate that simple but problem-specific features extracted from spectroscopic data can significantly improve the classification performance. Since our approach works orthogonal to existing classification schemes used for building the spectroscopic catalogs, our classification results are well …
We present the use of continuous-time autoregressive moving average (CARMA) models as a method for estimating the variability features of a light curve, and in particular its power spectral density (PSD). CARMA models fully account for irregular sampling and measurement errors, making them valuable for quantifying variability, forecasting and interpolating light curves, and for variability-based classification. We show that the PSD of a CARMA model can be expressed as a sum of Lorentzian functions, which makes them extremely flexible and able to model a broad range of PSDs. We present the like…
Technical University of Cluj-Napoca（克卢日-纳波卡技术大学）；Isaac Newton Group of Telescopes (ING)（艾萨克·牛顿望远镜组）；Instituto de Astrofisica de Canarias (IAC)（加那利天体物理研究所）
The continuing monitoring and surveying of the nearby space to detect Near Earth Objects (NEOs) and Near Earth Asteroids (NEAs) are essential because of the threats that this kind of objects impose on the future of our planet. We need more computational resources and advanced algorithms to deal with the exponential growth of the digital cameras' performances and to be able to process (in near real-time) data coming from large surveys. This paper presents a software platform called NEARBY that supports automated detection of moving sources (asteroids) among stars from astronomical images. The d…
Though the ultraviolet (UV) domain plays a vital role in the studies of astronomical transient events, the UV time-domain sky remains largely unexplored. We have designed a wide-field UV imager that can be flown on a range of available platforms, such as high-altitude balloons, CubeSats, and larger space missions. The major scientific goals are the variability of astronomical sources, detection of transients such as supernovae, novae, tidal disruption events, and characterizing AGN variability. The instrument has an 80 mm aperture with a circular field of view of 10.8 degrees, an angular resol…
Exoplanets, short for `extra solar planets', are planets outside our solar system. They are objects with masses less than around 15 Jupiter-masses that orbit stars other than the Sun. They are small enough so they can not burn deuterium in their cores, yet large enough that they are not so called `dwarf planets' like Pluto. To discover life elsewhere in the universe, particularly outside our own solar system, a good starting point would be to search for planets orbiting nearby Sun-like stars, since the only example of life we know of thrives on a planet we call Earth that orbits a G-type dwarf…
The next generation of telescopes will acquire terabytes of image data on a nightly basis. Collectively, these large images will contain billions of interesting objects, which astronomers call sources. The astronomers' task is to construct a catalog detailing the coordinates and other properties of the sources. The source catalog is the primary data product for most telescopes and is an important input for testing new astrophysical theories, but to construct the catalog one must first detect the sources. Existing algorithms for catalog creation are effective at detecting sources, but do not ha…
"exoplanet" is a toolkit for probabilistic modeling of astronomical time series data, with a focus on observations of exoplanets, using PyMC3 (Salvatier et al., 2016). PyMC3 is a flexible and high-performance model-building language and inference engine that scales well to problems with a large number of parameters. "exoplanet" extends PyMC3's modeling language to support many of the custom functions and probability distributions required when fitting exoplanet datasets or other astronomical time series. While it has been used for other applications, such as the study of stellar variability, t…
While fully automated methods for detecting faint moving objects in astronomical images - such as Kuiper belt objects (KBOs) - are constantly improving, visual detection still has a role to play especially when the fixed background is cluttered with stars. Color intensity projections (CIPs) using hue cycling - which combines a sequence of greyscale images into a single color image - aids in the visual detection of moving objects by highlighting them using color in an intuitive way. To demonstrate the usefulness of CIPs in detecting faint moving objects a sequence of 16 images from the SuprimeC…
A probabilistic technique for the joint estimation of background and sources with the aim of detecting faint and extended celestial objects is described. Bayesian probability theory is applied to gain insight into the coexistence of background and sources through a probabilistic two-component mixture model, which provides consistent uncertainties of background and sources. A multi-resolution analysis is used for revealing faint and extended objects in the frame of the Bayesian mixture model. All the revealed sources are parameterized automatically providing source position, net counts, morphol…
This paper reviews the Stochastic Recurrent Neural Network (SRNN) as applied to the light curves of Active Galactic Nuclei by Sheng et al. (2022). Astronomical data have inherent limitations arising from telescope capabilities, cadence strategies, inevitable observing weather conditions, and current understanding of celestial objects. When applying machine learning methods, it is vital to understand the effects of data limitations on our analysis and ability to make inferences. We take Sheng et al. (2022) as a case study, and illustrate the problems and limitations encountered in implementing …
Powellsnakes is a Bayesian algorithm for detecting compact objects embedded in a diffuse background, and was selected and successfully employed by the Planck consortium in the production of its first public deliverable: the Early Release Compact Source Catalogue (ERCSC). We present the critical foundations and main directions of further development of PwS, which extend it in terms of formal correctness and the optimal use of all the available information in a consistent unified framework, where no distinction is made between point sources (unresolved objects), SZ clusters, single or multi-chan…
The importance of archival science increases significantly for astrophysical observatories as they mature and their archive holdings grow in size and complexity. Further enhancing the science return of archival data requires engaging a larger audience than the mission reference community, mostly because of the growth of interest in multi-wavelength and transient/time variability research. Such a goal, though, can be difficult to achieve. In this paper I will describe a different approach to this question that, while minimizing technological friction and leveraging existing services, makes arch…
The AGILE space mission (whose instrument is sensitive in the energy ranges 18-60 keV, and 30 MeV - 50 GeV) has been operating since 2007. Assessing the statistical significance of time variability of gamma-ray sources above 100 MeV is a primary task of the AGILE data analysis. In particular, it is important to check the instrument sensitivity in terms of Poisson modeling of the data background, and to determine the post-trial confidence of detections. The goals of this work are: (i) evaluating the distributions of the likelihood ratio test for "empty" fields, and for regions of the Galactic p…
Istituto di Radio Astronomia, INAF（意大利国家天体物理研究所射电天文研究所）；Hamburger Sternwarte（汉堡天文台）；Universitá di Bologna（博洛尼亚大学）
We apply a Machine Learning technique known as Convolutional Denoising Autoencoder to denoise synthetic images of state-of-the-art radio telescopes, with the goal of detecting the faint, diffused radio sources predicted to characterise the radio cosmic web. In our application, denoising is intended to address both the reduction of random instrumental noise and the minimisation of additional spurious artefacts like the sidelobes, resulting from the aperture synthesis technique. The effectiveness and the accuracy of the method are analysed for different kinds of corrupted input images, together …
We investigate the performance of the generalized Spectral Kurtosis (SK) estimator in detecting and discriminating natural and artificial very short duration transients in the 2-bit sampling time domain Very-Long-Baseline Interferometry (VLBI) data. We demonstrate that, while both types of transients may be efficiently detected, their natural or artificial nature cannot be distinguished if only a time domain SK analysis is performed. However, these two types of transients become distinguishable from each other in the spectral domain, after a 32-bit FFT operation is performed on the 2-bit time …
Most major discoveries in astronomy are unplanned, and result from surveying the Universe in a new way, rather than by testing a hypothesis or conducting an investigation with planned outcomes. For example, of the 10 greatest discoveries made by the Hubble Space Telescope, only one was listed in its key science goals. So a telescope that merely achieves its stated science goals is not achieving its potential scientific productivity. Several next-generation astronomical survey telescopes are currently being designed and constructed that will significantly expand the volume of observational para…
The variance and fractional variance on a fixed time window (variously known as "rms percent" or "modulation index") are commonly used to characterize the variability of astronomical sources. We summarize properties of this statistic for a Gaussian process.
Dome A, Antarctica has been thought to be one of the best astronomical sites on the earth since decades ago. From it was first visited by astronomers in 2008, dozens of facilities for astronomical observation and site testing were deployed. Due to its special geographical location, the data and message exchange between Dome A and the domestic control center could only depend on Iridium. Because the link bandwidth of Iridium is extremely limited, meanwhile the network traffic cost is quite expensive and the network is rather unstable, the commonly used data transfer tools, such as rsync and scp…
A relatively unexplored phase space of transients and stellar variability is that of second and sub-second time-scales. We describe a new optical observatory operating in the Negev desert in Israel, with a 55 cm aperture, a field of view of 2.6x2.6 deg (~7deg^2) equipped with a high frame rate, low read noise, CMOS camera. The system can observe at a frame rate of up to 90HZ (full frame), while nominally observations are conducted at 10-25Hz. The data, generated at a rate of over 6Gbits/s at a frame rate of 25Hz, are analyzed in real time. The observatory is fully robotic and capable of autono…
Recent all-sky and large-area astronomical surveys and their catalogued data over the whole range of electromagnetic spectrum are reviewed, from Gamma-ray to radio, such as Fermi-GLAST and INTEGRAL in Gamma-ray, ROSAT, XMM and Chandra in X-ray, GALEX in UV, SDSS and several POSS I and II based catalogues (APM, MAPS, USNO, GSC) in optical range, 2MASS in NIR, WISE and AKARI IRC in MIR, IRAS and AKARI FIS in FIR, NVSS and FIRST in radio and many others, as well as most important surveys giving optical images (DSS I and II, SDSS, etc.), proper motions (Tycho, USNO, Gaia), variability (GCVS, NSVS,…
Our understanding of the Universe has profited from deliberate, targeted studies of known phenomena, as well as from serendipitous, unexpected discoveries, such as the discovery of a complex variability pattern in the direction of KIC 8462852 (Boyajian's star). Upcoming surveys, such as the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), will explore the parameter space of astrophysical transients at all time scales, and offer the opportunity to discover even more extreme examples of unexpected phenomena. We investigate strategies to identify novel objects and to contextualiz…
The reliability of detecting source variability in sparsely and irregularly sampled X-ray light curves is investigated. This is motivated by the unprecedented survey capabilities of eROSITA onboard SRG, providing light curves for many thousand sources in its final-depth equatorial deep field survey. Four methods for detecting variability are evaluated: excess variance, amplitude maximum deviations, Bayesian blocks and a new Bayesian formulation of the excess variance. We judge the false detection rate of variability based on simulated Poisson light curves of constant sources, and calibrate sig…
Nanjing University of Posts and Telecommunications（南京邮电大学）；National Astronomical Observatories, Chinese Academy of Sciences（中国科学院国家天文台）；University of Chinese Academy of Sciences（中国科学院大学）
Astronomical outliers, such as unusual, rare or unknown types of astronomical objects or phenomena, constantly lead to the discovery of genuinely unforeseen knowledge in astronomy. More unpredictable outliers will be uncovered in principle with the increment of the coverage and quality of upcoming survey data. However, it is a severe challenge to mine rare and unexpected targets from enormous data with human inspection due to a significant workload. Supervised learning is also unsuitable for this purpose since designing proper training sets for unanticipated signals is unworkable. Motivated by…
This article presents a novel type of very long baseline astronomical interferometer that uses the fluctuations, as a function of time, of the intensity measured by a quadratic detector, which is a common type of astronomical detector. The theory on which the technique is based is validated by laboratory experiments. Its outstanding principal advantages comes from the fact that the angular structure of an astronomical object is simply determined from the visibility of the minima of the spectrum of the intensity fluctuations measured by the detector, as a function of the frequency of the fluctu…
AstroSkyFlow: an astronomical sky image flow simulator for time domain survey validation and machine learning
State Key Laboratory of Dark Matter Physics, Tsung-Dao Lee Institute & School of Physics and Astronomy, Shanghai Jiao Tong University（暗物质物理全国重点实验室，李政道研究所，上海交通大学物理与天文学院）；Shanghai Astronomical Observatory, Chinese Academy of Sciences（中国科学院上海天文台）
The Millennium Institute of Astrophysics（千年天体物理研究所）；Pontificia Universidad Católica de Chile（智利天主教大学）；Harvard University（哈佛大学）；Harvard-Smithsonian Center for Astrophysics（哈佛-史密森天体物理中心）；Max-Planck Institute for Astronomy（马克斯·普朗克天文研究所）
The development of synoptic sky surveys has led to a massive amount of data for which resources needed for analysis are beyond human capabilities. To process this information and to extract all possible knowledge, machine learning techniques become necessary. Here we present a new method to automatically discover unknown variable objects in large astronomical catalogs. With the aim of taking full advantage of all the information we have about known objects, our method is based on a supervised algorithm. In particular, we train a random forest classifier using known variability classes of objec…
This paper introduces the multiband periodogram, a general extension of the well-known Lomb-Scargle approach for detecting periodic signals in time-domain data. In addition to advantages of the Lomb-Scargle method such as treatment of non-uniform sampling and heteroscedastic errors, the multiband periodogram significantly improves period finding for randomly sampled multiband light curves (e.g., Pan-STARRS, DES and LSST). The light curves in each band are modeled as arbitrary truncated Fourier series, with the period and phase shared across all bands. The key aspect is the use of Tikhonov regu…
In the preparation for ESA's Euclid mission and the large amount of data it will produce, we train deep convolutional neural networks on Euclid simulations classify solar system objects from other astronomical sources. Using transfer learning we are able to achieve a good performance despite our tiny dataset with as few as 7512 images. Our best model correctly identifies objects with a top accuracy of 94% and improves to 96% when Euclid's dither information is included. The neural network misses ~50% of the slowest moving asteroids (v < 10 arcsec/h) but is otherwise able to correctly classify …
Variable emission is one of the defining characteristic of active galactic nuclei (AGN). While providing precious information on the nature and physics of the sources, variability is often challenging to observe with time- and field-of-view-limited astronomical observatories such as Imaging Atmospheric Cherenkov Telescopes (IACTs). In this work, we address two questions relevant for the observation of sources characterized by AGN-like variability: what is the most time-efficient way to detect such sources, and what is the observational bias that can be introduced by the choice of the observing…
This paper introduces an extension of the linear least-squares (or Lomb-Scargle) periodogram for the case when the model of the signal to be detected is non-sinusoidal and depends on unknown parameters in a non-linear manner. The attention is paid to the problem of estimating the statistical significance of candidate periodicities found using such non-linear periodograms. This problem is related to the task of quantifying the distributions of maximum values of these periodograms. Based on recent results in the mathematical theory of extreme values of random field (the generalized Rice method),…
Coloring Black Holes: Epistemic and Aesthetic Choices in Astronomical Imaging
Institute of Cultural Anthropology and Development Sociology, Leiden University（莱顿大学文化人类学与发展社会学研究所）；Niels Bohr Archive, University of Copenhagen（哥本哈根大学尼尔斯·玻尔档案馆）；Institute for Theoretical Physics Amsterdam, University of Amsterdam（阿姆斯特丹大学理论物理研究所）；Vossius Center for the History of Humanities and Sciences, University of Amsterdam（阿姆斯特丹大学沃斯修斯人文与科学历史中心）
Celestial objects exhibit a wide range of variability in brightness at different wavebands. Surprisingly, the most common methods for characterizing time series in statistics -- parametric autoregressive modeling -- is rarely used to interpret astronomical light curves. We review standard ARMA, ARIMA and ARFIMA (autoregressive moving average fractionally integrated) models that treat short-memory autocorrelation, long-memory $1/f^α$ `red noise', and nonstationary trends. Though designed for evenly spaced time series, moderately irregular cadences can be treated as evenly-spaced time series wit…
Time series observations are ubiquitous in astronomy, and are generated to distinguish between different types of supernovae, to detect and characterize extrasolar planets and to classify variable stars. These time series are usually modeled using a parametric and/or physical model that assumes independent and homoscedastic errors, but in many cases these assumptions are not accurate and there remains a temporal dependency structure on the errors. This can occur, for example, when the proposed model cannot explain all the variability of the data or when the parameters of the model are not prop…
In recent years, the cross spectrum has received considerable attention as a means of characterising the variability of astronomical sources as a function of wavelength. While much has been written about the statistics of time and phase lags, the cospectrum has only recently been understood as means of mitigating instrumental effects dependent on temporal frequency in astronomical detectors, as well as a method of characterizing the coherent variability in two wavelength ranges on different time scales. In this paper, we lay out the statistical foundations of the cospectrum, starting with the …
This paper addresses the problem of detecting and characterizing local variability in time series and other forms of sequential data. The goal is to identify and characterize statistically significant variations, at the same time suppressing the inevitable corrupting observational errors. We present a simple nonparametric modeling technique and an algorithm implementing it - an improved and generalized version of Bayesian Blocks (Scargle 1998) - that finds the optimal segmentation of the data in the observation interval. The structure of the algorithm allows it to be used in either a real-time…
Time series data mining is an important field of research in the era of "Big Data". Next generation astronomical surveys will generate data at unprecedented rates, creating the need for automated methods of data analysis. We propose a method of light curve characterisation that employs a pipeline consisting of a neural network with a Long-Short Term Memory Variational Autoencoder architecture and a Gaussian mixture model. The pipeline performs extraction and aggregation of features from light curve segments into feature vectors of fixed length which we refer to as light curve "fingerprints". T…
Kavli Institute for Particle Astrophysics and Cosmology（卡弗里粒子天体物理与宇宙学研究所）；Stanford University（斯坦福大学）；SLAC National Accelerator Laboratory（SLAC国家加速器实验室）；Flatiron Institute（熨斗研究所）；Columbia University（哥伦比亚大学）；Princeton University（普林斯顿大学）；New York University（纽约大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Among the most extreme objects in the Universe, active galactic nuclei (AGN) are luminous centers of galaxies where a black hole feeds on surrounding matter. The variability patterns of the light emitted by an AGN contain information about the physical properties of the underlying black hole. Upcoming telescopes will observe over 100 million AGN in multiple broadband wavelengths, yielding a large sample of multivariate time series with long gaps and irregular sampling. We present a method that reconstructs the AGN time series and simultaneously infers the posterior probability density distribu…
