We present a methodology for automated real-time analysis of a radio image data stream with the goal to find transient sources. Contrary to previous works, the transients we are interested in occur on a time-scale where dispersion starts to play a role, so we must search a higher-dimensional data space and yet work fast enough to keep up with the data stream in real time. The approach consists of five main steps: quality control, source detection, association, flux measurement, and physical parameter inference. We present parallelized methods based on convolutions and filters that can be accelerated on a GPU, allowing the pipeline to run in real-time. In the parameter inference step, we apply a convolutional neural network to dynamic spectra that were obtained from the preceding steps. It infers physical parameters, among which the dispersion measure of the transient candidate. Based on critical values of these parameters, an alert can be sent out and data will be saved for further investigation. Experimentally, the pipeline is applied to simulated data and images from AARTFAAC (Amsterdam Astron Radio Transients Facility And Analysis Centre), a transients facility based on the Low-Frequency Array (LOFAR). Results on simulated data show the efficacy of the pipeline, and from real data it discovered dispersed pulses. The current work targets transients on time scales that are longer than the fast transients of beam-formed search, but shorter than slow transients in which dispersion matters less. This fills a methodological gap that is relevant for the upcoming Square-Kilometer Array (SKA). Additionally, since real-time analysis can be performed, only data with promising detections can be saved to disk, providing a solution to the big-data problem that modern astronomy is dealing with.
Searching for dispersed radio pulses in interferometric data is of great scientific interest, but poses a formidable computational burden. Here we present two efficient, new antenna-coherent solutions: The Chirpolator and The Chimageator. We describe the equations governing both techniques and propose a number of novel optimizations. We compare the implementation costs of our techniques with classical methods using three criteria: the operations rates (1) before and (2) after the integrate-and-dump stage, and (3) the data rate directly after the integrate-and-dump stage. When compared with cla…
We demonstrate a new technique for detecting radio transients based on interferometric closure quantities. The technique uses the bispectrum, the product of visibilities around a closed-loop of baselines of an interferometer. The bispectrum is calibration independent, resistant to interference, and computationally efficient, so it can be built into correlators for real-time transient detection. Our technique could find celestial transients anywhere in the field of view and localize them to arcsecond precision. At the Karl G. Jansky Very Large Array (VLA), such a system would have a high survey…
Recently, there have been reports of six bright, dispersed bursts of coherent radio emission found in pulsar surveys with the Parkes Multi-beam Receiver. Not much is known about the progenitors of these bursts, but they are highly-energetic, and probably of extragalactic origin. Their properties suggest extreme environments and interesting physics, but in order to understand and study these events, more examples need to be found. Fortunately, the recent boom in radio astronomy means many 'next-generation' radio telescopes are set to begin observing in the near future. In this paper we discuss …
Computationally inexpensive algorithm for detecting of dispersed transients has been developed using Cumulative Sums (CUSUM) scheme for detecting abrupt changes in statistical characteristics of the signal. The efficiency of the algorithm is demonstrated on pulsar PSR J0835-4510.
Isolated, short dispersed pulses of radio emission of unknown origin have been reported and there is strong interest in wide-field, sensitive searches for such events. To achieve high sensitivity, large collecting area is needed and dispersion due to the interstellar medium should be removed. To survey a large part of the sky in reasonable time, a telescope that forms multiple simultaneous beams is desirable. We have developed a novel FPGA-based transient search engine that is suitable for these circumstances. It accepts short-integration-time spectral power measurements from each beam of the …
Radio transients are sporadic signals and their detection requires that the backends of radio telescopes be equipped with the appropriate hardware and software to undertake this. Observational programs to detect transients can be dedicated or they can piggy-back on observations made by other programs. It is the single-dish single-transient (non-periodical) mode which is considered in this paper. Because neither the width of a transient nor the time of its arrival is known, a sequential analysis in the form of a cumulative sum (cusum) algorithm is proposed here. Computer simulations and real ob…
We demonstrate a signal processing concept for imaging the sky at millisecond rates with radio interferometers. The "Pocket Correlator" (PoCo) correlates the signals from multiple elements of a radio interferometer fast enough to image brief, dispersed pulses. By the nature of interferometry, a millisecond correlator functions like a large, single-dish telescope, but with improved survey speed, spatial localization, calibration, and interference rejection. To test the concept, we installed PoCo at the Allen Telescope Array (ATA) to search for dispersed pulses from the Crab pulsar, B0329+54, an…
The Australian SKA Pathfinder (ASKAP) is a next generation radio telescope currently under construction in Western Australia. The fast survey speed and wide field of view make it an ideal instrument for blind transients searches. The ASKAP Variable and Slow Transients (VAST) survey is a one of the major science programs planned for ASKAP. The scientific goals of VAST include the detection and characterisation of a wide range of transient and variable phenomena, from gamma-ray burst afterglows to extreme scattering events, on timescales of 5 seconds or longer. We describe the data and processin…
Searching for extraterrestrial, transient signals in astronomical data sets is an active area of current research. However, machine learning techniques are lacking in the literature concerning single-pulse detection. This paper presents a new, two-stage approach for identifying and classifying dispersed pulse groups (DPGs) in single-pulse search output. The first stage identified DPGs and extracted features to characterize them using a new peak identification algorithm which tracks sloping tendencies around local maxima in plots of signal-to-noise ratio vs. dispersion measure. The second stage…
There is a broad class of astrophysical sources that produce detectable, transient, gravitational waves. Some searches for transient gravitational waves are tailored to known features of these sources. Other searches make few assumptions about the sources. Typically events are observable with multiple search techniques. This work describes how to combine the results of searches that are not independent, treating each search as a classifier for a given event. This will be shown to improve the overall sensitivity to gravitational-wave events while directly addressing the problem of consistent in…
As second-generation gravitational-wave detectors prepare to analyze data at unprecedented sensitivity, there is great interest in searches for unmodeled transients, commonly called bursts. Significant effort has yielded a variety of techniques to identify and characterize such transient signals, and many of these methods have been applied to produce astrophysical results using data from first-generation detectors. However, the computational cost of background estimation remains a challenging problem; it is difficult to claim a 5σ detection with reasonable computational resources without payin…
We explore the possibility of very long-lived gravitational-wave transients (and detector artifacts) lasting hours to weeks. Such very long signals are both interesting in their own right and as a potential source of systematic error in searches for persistent signals, e.g., from a stochastic gravitational-wave background. We review possible mechanisms for emission on these time scales and discuss computational challenges associated with their detection: namely, the substantial volume of data involved in a search for very long transients can require vast computer memory and processing time. Th…
There are several unexplored regions of the short-duration radio transient phase space. One such unexplored region is the luminosity gap between giant pulses (from pulsars) and cosmologically located fast radio bursts (FRBs). The Survey for Transient Astronomical Radio Emission 2 (STARE2) is a search for such transients out to 7 Mpc. STARE2 has a field of view of 3.6 steradians and is sensitive to one-millisecond transients above $\sim300$ kJy. With a two-station system we have detected and localized a solar burst, demonstrating that the pilot system is capable of detecting short duration radi…
We present and evaluate several strategies to search for prompt, low-frequency radio emission associated with gravitational wave transients using the Murchison Widefield Array (MWA). As we are able to repoint the MWA on timescales of tens of seconds, we can search for the dispersed radio signal that has been predicted to originate along with or shortly after a neutron star-neutron star merger. We find that given the large, 600 deg^2 instantaneous field-of-view of the MWA we can cover a significant fraction of the predicted gravitational wave error region, although due to the complicated geomet…
The University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德河谷分校）；University of Warsaw（华沙大学）；National Centre for Nuclear Research（国家核研究中心）；Texas Tech University（德克萨斯理工大学）
To search for optical counterparts to gravitational waves, it is crucial to develop an efficient follow-up method that allows for both a quick telescopic scan of the event localization region and search through the resulting image data for plausible optical transients. We present a method to detect these transients based on an artificial neural network. We describe the architecture of two networks capable of comparing images of the same part of the sky taken by different telescopes. One image corresponds to the epoch in which a potential transient could exist; the other is a reference image of…
Transient phenomena are interesting and potentially highly revealing of details about the processes under observation and study that could otherwise go unnoticed. It is therefore important to maximize the sensitivity of the method used to identify such events. In this article, we present a general procedure based on the use of the likelihood function for identifying transients which is particularly suited for real-time applications because it requires no grouping or pre-processing of the data. The method makes use of all the information that is available in the data throughout the statistical …
Recent discoveries of dispersed, non-periodic impulsive radio signals with single-dish radio telescopes have sparked significant interest in exploring the relatively uncharted space of fast transient radio signals. Here we describe V-FASTR, an experiment to perform a blind search for fast transient radio signals using the Very Long Baseline Array (VLBA). The experiment runs entirely in a commensal mode, alongside normal VLBA observations and operations. It is made possible by the features and flexibility of the DiFX software correlator that is used to process VLBA data. Using the VLBA for this…
SAINT (Small Aperture Imaging Network Telescope) -- a wide-field telescope complex for detecting and studying optical transients at times from milliseconds to years
We present Clusterrank, a new algorithm for identifying dispersed astrophysical pulses. Such pulses are commonly detected from Galactic pulsars and rotating radio transients (RRATs), which are neutron stars with sporadic radio emission. More recently, isolated, highly dispersed pulses dubbed fast radio bursts (FRBs) have been identified as the potential signature of an extragalactic cataclysmic radio source distinct from pulsars and RRATs. Clusterrank helped us discover 14 pulsars and 8 RRATs in data from the Arecibo 327 MHz Drift Pulsar Survey (AO327). The new RRATs have DMs in the range $23.…
Rotating radio transients (RRATs) are sporadically emitting pulsars detectable only through searches for single pulses. While over 100 RRATs have been detected, only a small fraction (roughly 20\%) have phase-connected timing solutions, which are critical for determining how they relate to other neutron star populations. Detecting more pulses in order to achieve solutions is a key to understanding their physical nature. Astronomical signals collected by radio telescopes contain noise from many sources, making the detection of weak pulses difficult. Applying a denoising method to raw time serie…
We have performed an in-depth concept study of a gravitational wave data analysis method which targets repeated long quasi-monochromatic transients (triggers) from cosmic sources. The algorithm concept can be applied to multi-trigger data sets in which the detector-source orientation and the statistical properties of the data stream change with time, and does not require the assumption that the data is Gaussian. Reconstructing or limiting the energetics of potential gravitational wave emissions associated with quasi-periodic oscillations (QPOs) observed in the X-ray lightcurve tails of soft ga…
Imaging by aperture synthesis from interferometric data is a well-known, but is a strong ill-posed inverse problem. Strong and faint radio sources can be imaged unambiguously using time and frequency integration to gather more Fourier samples of the sky. However, these imagers assumes a steady sky and the complexity of the problem increases when transients radio sources are also present in the data. Hopefully, in the context of transient imaging, the spatial and temporal information are separable which enable extension of an imager fit for a steady sky. We introduce independent spatial and tem…
We investigate the optimal tradeoff between sensitivity and field of view in surveys for slow radio transients using the event detection rate as the survey metric. We investigate (i) a survey in which the events are distributed homogeneously throughout a volume centred on the Earth, (ii) a survey in which the events are homogeneously distributed, but are only detectable beyond a certain minimum distance, and (iii) a survey in which all the events occur at an identical distance, as is appropriate for a targetted survey of a particular field which subtends N_point telescope pointings. For a surv…
The detection of the gravitational wave counterpart GRB 170817A, underluminous compared to the cosmological GRB population by a factor of 10,000, motivates significant effort in detecting and localizing a dim, nearby, and slightly off-axis population of short GRBs. Swift/BAT is the most sensitive GRB detector in operation, and the only one that regularly localizes GRBs to arcminute precision, critical to rapid followup studies. However, the utility of BAT in targeted sub-threshold searches had been historically curtailed by the unavailability of the necessary raw data for analysis. The new ava…
The AGILE space mission (whose instrument is sensitive in the energy ranges 18-60 keV, and 30 MeV - 50 GeV) has been operating since 2007. Assessing the statistical significance of time variability of gamma-ray sources above 100 MeV is a primary task of the AGILE data analysis. In particular, it is important to check the instrument sensitivity in terms of Poisson modeling of the data background, and to determine the post-trial confidence of detections. The goals of this work are: (i) evaluating the distributions of the likelihood ratio test for "empty" fields, and for regions of the Galactic p…
In the multi-messenger era, facilities share their results with the scientific community through networks such as the General Coordinates Network to study transient phenomena (e.g., Gamma-ray bursts) and implement real-time analysis pipelines to detect transient events, reacting to science alerts received from other observatories. The fast analysis of transient events is crucial for detecting counterparts of gravitational waves and neutrino candidate events. In this context, collecting scientific results from different high-energy satellites observing the same transient event represents a key …
Radio interferometers have the ability to precisely localize and better characterize the properties of sources. This ability is having a powerful impact on the study of fast radio transients, where a few milliseconds of data is enough to pinpoint a source at cosmological distances. However, recording interferometric data at millisecond cadence produces a terabyte-per-hour data stream that strains networks, computing systems, and archives. This challenge mirrors that of other domains of science, where the science scope is limited by the computational architecture as much as the physical process…
We present the design and first results of a real-time search for transients within the 650 sq. deg. area around the Magellanic Clouds, conducted as part of the OGLE-IV project and aimed at detecting supernovae, novae and other events. The average sampling of about 4 days from September to May, yielded a detection of 238 transients in 2012/2013 and 2013/2014 seasons. The superb photometric and astrometric quality of the OGLE data allows for numerous applications of the discovered transients. We use this sample to prepare and train a Machine Learning-based automated classifier for early light c…
We investigate the performance of the generalized Spectral Kurtosis (SK) estimator in detecting and discriminating natural and artificial very short duration transients in the 2-bit sampling time domain Very-Long-Baseline Interferometry (VLBI) data. We demonstrate that, while both types of transients may be efficiently detected, their natural or artificial nature cannot be distinguished if only a time domain SK analysis is performed. However, these two types of transients become distinguishable from each other in the spectral domain, after a 32-bit FFT operation is performed on the 2-bit time …
Next-generation radio arrays, including the Square Kilometre Array (SKA) and its pathfinders, will open up new avenues for exciting transient science at radio wavelengths. Their innovative designs, comprising a large number of small elements, pose several challenges in digital processing and optimal observing strategies. The Giant Metrewave Radio Telescope (GMRT) presents an excellent test-bed for developing and validating suitable observing modes and strategies for transient experiments with future arrays. Here we describe the first phase of the ongoing development of a transient detection sy…
Fast Radio Bursts (FRBs) are short duration highly energetic dispersed radio pulses. We developed a generic formalism (Bera {\em et al.}, 2016) to estimate the FRB detection rate for any radio telescope with given parameters. By using this model, we estimated the FRB detection rate for two Indian radio telescope; the Ooty Wide Field Array (OWFA) (Bhattacharyya {\em et al.}, 2017) and the upgraded Giant Metrewave Radio Telescope (uGMRT) (Bhattacharyya {\em et al.}, 2018) with three beam forming modes. In this review article, I summarize these two works. We considered the energy spectrum of FRBs…
Columbia University in the City of New York（纽约市哥伦比亚大学）；University of Florida（佛罗里达大学）
As engineered systems grow in complexity, there is an increasing need for automatic methods that can detect, diagnose, and even correct transient anomalies that inevitably arise and can be difficult or impossible to diagnose and fix manually. Among the most sensitive and complex systems of our civilization are the detectors that search for incredibly small variations in distance caused by gravitational waves -- phenomena originally predicted by Albert Einstein to emerge and propagate through the universe as the result of collisions between black holes and other massive objects in deep space. T…
Data from current gravitational wave detectors contains a high rate of transient noise (glitches) that can trigger false detections and obscure true astrophysical events. Existing noise-detection algorithms largely rely on model-based methods that may miss noise transients unwitnessed by auxiliary sensors or with exotic morphologies. We propose the Unicorn Multi-window Anomaly-detection Pipeline (UniMAP): a model-free algorithm to identify and characterize transient noise leveraging the Temporal Outlier Factor (TOF) via a multi-window data-resampling scheme. We show this windowing scheme exten…
Standard detection and analysis techniques for transient gravitational waves make the assumption that detector data contains, at most, one signal at any time. As detectors improve in sensitivity, this assumption will no longer be valid. In this paper we examine how current search techniques for transient gravitational waves will behave under the presence of more than one signal. We perform searches on data sets containing time-overlapping compact binary coalescences. This includes a modelled, matched filter search (PyCBC), and an unmodelled coherent search, coherent WaveBurst (cWB). Both of th…
Pulsar searches are computationally demanding efforts to discover dispersed periodic signals in time- and frequency-resolved data from radio telescopes. The complexity and computational expense of simultaneously determining the frequency-dependent delay (dispersion) and the periodicity of the signal is further exacerbated by the presence of various types of radio-frequency interference (RFI) and observing-system effects. New observing systems with wider bandwidths, higher bit rates and greater overall sensitivity (also to RFI) further enhance these challenges. We present a novel approach to th…
Recent discoveries of highly dispersed millisecond radio bursts by Thornton et al. in a survey with the Parkes radio telescope at 1.4 GHz point towards an emerging population of sources at cosmological distances whose origin is currently unclear. Here we demonstrate that the scattering effects at lower radio frequencies are less than previously thought, and that the bursts could be detectable at redshifts out to about $z=0.5$ in surveys below 1 GHz. Using a source model in which the bursts are standard candles with bolometric luminosities $\sim 8 \times 10^{44}$ ergs/s uniformly distributed pe…
We are developing a purely commensal survey experiment for fast (<5s) transient radio sources. Short-timescale transients are associated with the most energetic and brightest single events in the Universe. Our objective is to cover the enormous volume of transients parameter space made available by ASKAP, with an unprecedented combination of sensitivity and field of view. Fast timescale transients open new vistas on the physics of high brightness temperature emission, extreme states of matter and the physics of strong gravitational fields. In addition, the detection of extragalactic objects af…
A signal dedispersion algorithm for imaging-based transient searches
CSIRO（澳大利亚联邦科学与工业研究组织）；Pawsey Supercomputing Research Centre（珀斯超级计算研究中心）；International Centre for Radio Astronomy Research, Curtin University（科廷大学国际射电天文研究中心）；Australian SKA Regional Centre (AusSRC), The University of Western Australia（西澳大利亚大学澳大利亚SKA区域中心）；Square Kilometre Array Observatory (SKAO)（平方公里阵列射电望远镜观测站）
We describe ZStreak, a semi-real-time pipeline specialized in detecting small, fast-moving near-Earth asteroids (NEAs) that is currently operating on the data from the newly-commissioned Zwicky Transient Facility (ZTF) survey. Based on a prototype originally developed by Waszczak et al. (2017) for the Palomar Transient Factory (PTF), the predecessor of ZTF, ZStreak features an improved machine-learning model that can cope with the $10\times$ data rate increment between PTF and ZTF. Since its first discovery on 2018 February 5 (2018 CL), ZTF/ZStreak has discovered $45$ confirmed new NEAs over a…
The Global Network of Optical Magnetometers for Exotic physics searches (GNOME) is a network of time-synchronized, geographically separated, optically pumped atomic magnetometers that is being used to search for correlated transient signals heralding exotic physics. GNOME is sensitive to exotic couplings of atomic spins to certain classes of dark matter candidates, such as axions. This work presents a data analysis procedure to search for axion dark matter in the form of topological defects: specifically, walls separating domains of discrete degenerate vacua in the axion field. An axion domain…
University of Cambridge（剑桥大学）；Massachusetts Institute of Technology（麻省理工学院）；The Alan Turing Institute（艾伦·图灵研究所）；University of the Western Cape（西开普大学）；South African Radio Astronomy Observatory（南非射电天文台）；African Institute for Mathematical Sciences（非洲数学科学研究所）；Swinburne University of Technology（斯威本科技大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
New time-domain surveys, such as the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), will observe millions of transient alerts each night, making standard approaches of visually identifying new and interesting transients infeasible. We present two novel methods of automatically detecting anomalous transient light curves in real-time. Both methods are based on the simple idea that if the light curves from a known population of transients can be accurately modelled, any deviations from model predictions are likely anomalies. The first modelling approach is a probabilistic neura…
Searches for gravitational waves crucially depend on exact signal processing of noisy strain data from gravitational wave detectors, which are known to exhibit significant non-Gaussian behavior. In this paper, we study two distinct non-Gaussian effects in the LIGO/Virgo data which reduce the sensitivity of searches: first, variations in the noise power spectral density (PSD) on timescales of more than a few seconds; and second, loud and abrupt transient `glitches' of terrestrial or instrumental origin. We derive a simple procedure to correct, at first order, the effect of the variation in the …
