John D. Swinbank, Tim D. Staley, Gijs J. Molenaar, Evert Rol, Antonia Rowlinson, Bart Scheers, Hanno Spreeuw, Martin E. Bell, Jess W. Broderick, Dario Carbone, … 展开作者
John D. Swinbank, Tim D. Staley, Gijs J. Molenaar, Evert Rol, Antonia Rowlinson, Bart Scheers, Hanno Spreeuw, Martin E. Bell, Jess W. Broderick, Dario Carbone, Alexander J. van der Horst, Casey J. Law, Michael Wise, Rene P. Breton, Yvette Cendes, Stéphane Corbel, Jochen Eislöffel, Heino Falcke, Rob Fender, Jean-Mathias Greißmeier, Jason W. T. Hessels, Benjamin W. Stappers, Adam J. Stewart, Ralph A. M. J. Wijers, Rudy Wijnands, Philippe Zarka
Current and future astronomical survey facilities provide a remarkably rich opportunity for transient astronomy, combining unprecedented fields of view with high sensitivity and the ability to access previously unexplored wavelength regimes. This is particularly true of LOFAR, a recently-commissioned, low-frequency radio interferometer, based in the Netherlands and with stations across Europe. The identification of and response to transients is one of LOFAR's key science goals. However, the large data volumes which LOFAR produces, combined with the scientific requirement for rapid response, make automation essential. To support this, we have developed the LOFAR Transients Pipeline, or TraP. The TraP ingests multi-frequency image data from LOFAR or other instruments and searches it for transients and variables, providing automatic alerts of significant detections and populating a lightcurve database for further analysis by astronomers. Here, we discuss the scientific goals of the TraP and how it has been designed to meet them. We describe its implementation, including both the algorithms adopted to maximize performance as well as the development methodology used to ensure it is robust and reliable, particularly in the presence of artefacts typical of radio astronomy imaging. Finally, we report on a series of tests of the pipeline carried out using simulated LOFAR observations with a known population of transients.
Modern radio interferometers such as the LOw Frequency ARray (LOFAR) are capable of producing data at hundreds of gigabits to terabits per second. This high data rate makes the analysis of radio data cumbersome and computationally expensive. While high performance computing facilities exist for large national and international facilities, that may not be the case for instruments operated by a single institution or a small consortium. Data rates for next generation radio telescopes are set to eclipse those currently in operation, hence local processing of data will become all the more important…
LOFAR is a leading aperture synthesis telescope operated in the Netherlands with stations across Europe. The LOFAR Two-meter Sky Survey (LoTSS) will produce more than 3000 14 TB data sets, mapping the entire northern sky at low frequencies. The data produced by this survey is important for understanding the formation and evolution of galaxies, supermassive black holes and other astronomical phenomena. All of the LoTSS data needs to be processed by the LOFAR Direction Independent (DI) pipeline, prefactor. Understanding the performance of this pipeline is important when trying to optimize the th…
Modern astronomical data processing requires complex software pipelines to process ever growing datasets. For radio astronomy, these pipelines have become so large that they need to be distributed across a computational cluster. This makes it difficult to monitor the performance of each pipeline step. To gain insight into the performance of each step, a performance monitoring utility needs to be integrated with the pipeline execution. In this work we have developed such a utility and integrated it with the calibration pipeline of the Low Frequency Array, LOFAR, a leading radio telescope. We te…
The new generation of high-resolution broad-band radio telescopes, like the Low Frequency Array (LOFAR), produces, depending on the level of compression, between 1 to 10 TB of data per hour after correlation. Such a large amount of scientific data demand powerful computing resources and efficient data handling strategies to be mastered. The LOFAR Two-metre Sky Survey (LoTSS) is a Key Science Project (KSP) of the LOFAR telescope. It aims to map the entire northern hemisphere at unprecedented sensitivity and resolution. The survey consist of 3 168 pointings, requiring about 30 PBytes of storage …
With the arrival of a number of wide-field snapshot image-plane radio transient surveys, there will be a huge influx of images in the coming years making it impossible to manually analyse the datasets. Automated pipelines to process the information stored in the images are being developed, such as the LOFAR Transients Pipeline, outputting light curves and various transient parameters. These pipelines have a number of tuneable parameters that require training to meet the survey requirements. This paper utilises both observed and simulated datasets to demonstrate different machine learning strat…
The Square Kilometre Array (SKA) will be both the largest radio telescope ever constructed and the largest Big Data project in the known Universe. The first phase of the project will generate on the order of 5 zettabytes of data per year. A critical task for the SKA will be its ability to process data for science, which will need to be conducted by science pipelines. Together with polarization data from the LOFAR Multifrequency Snapshot Sky Survey (MSSS), we have been developing a realistic SKA-like science pipeline that can handle the large data volumes generated by LOFAR at 150 MHz. The pipe…
Context: New generation low-frequency telescopes are exploring a new parameter space in terms of depth and resolution. The data taken with these interferometers, for example with the LOw Frequency ARray (LOFAR), are often calibrated in a low signal-to-noise ratio regime and the removal of critical systematic effects is challenging. The process requires an understanding of their origin and properties. Aim: In this paper we describe the major systematic effects inherent to next generation low-frequency telescopes, such as LOFAR. With this knowledge, we introduce a data processing pipeline that i…
The search for radio emission from exoplanets using LOFAR low-frequency beam-formed observations: Data pipeline and preliminary results for the 55 Cnc system
[abridged] The International LOFAR Telescope is an interferometer with stations spread across Europe. With baselines of up to ~2,000 km, LOFAR has the unique capability of achieving sub-arcsecond resolution at frequencies below 200 MHz, although this is technically and logistically challenging. Here we present a calibration strategy that builds on previous high-resolution work with LOFAR. We give an overview of the calibration strategy and discuss the special challenges inherent to enacting high-resolution imaging with LOFAR, and describe the pipeline, which is publicly available, in detail. W…
Searches for millisecond-duration, dispersed single pulses have become a standard tool used during radio pulsar surveys in the last decade. They have enabled the discovery of two new classes of sources: rotating radio transients and fast radio bursts. However, we are now in a regime where the sensitivity to single pulses in radio surveys is often limited more by the strong background of radio frequency interference (RFI, which can greatly increase the false-positive rate) than by the sensitivity of the telescope itself. To mitigate this problem, we introduce the Single-pulse Searcher (SpS). Th…
The LOw-Frequency ARray is a low-frequency radio interferometer composed by observational stations spread across Europe and it is the largest precursor of SKA in terms of effective area and generated data rates. In 2018, the Italian community officially joined LOFAR project, and it deployed a distributed computing and storage infrastructure dedicated to LOFAR data analysis. The infrastructure is based on 4 nodes distributed in different Italian locations and it offers services for pipelines execution, storage of final and intermediate results and support for the use of the software and infrast…
The Low Frequency Array (LOFAR) is an ideal instrument to conduct deep extragalactic surveys. It has a large field of view and is sensitive to large scale and compact emission. It is, however, very challenging to synthesize thermal noise limited maps at full resolution, mainly because of the complexity of the low-frequency sky and the direction dependent effects (phased array beams and ionosphere). In this first paper of a series we present a new calibration and imaging pipeline that aims at producing high fidelity, high dynamic range images with LOFAR High Band Antenna data, while being compu…
A GPU-Accelerated Transient Detection Pipeline for DECam Time-Domain Surveys
Carnegie Mellon University（卡内基梅隆大学）；University of Pennsylvania（宾夕法尼亚大学）；Texas A. & M. University（德克萨斯农工大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
We present VOLKS2, the second release of "VLBI Observation for transient Localization Keen Searcher". The pipeline aims at transient search in regular VLBI observations as well as detection of single pulses from known sources in dedicated VLBI observations. The underlying method takes the idea of geodetic VLBI data processing, including fringe fitting to maximize the signal power and geodetic VLBI solving for localization. By filtering the candidate signals with multiple windows within a baseline and by cross matching with multiple baselines, RFIs are eliminated effectively. Unlike the station…
Discovered in 2011 with LOFAR, the $15$ Jy low-frequency radio transient ILT J225347+862146 heralds a potentially prolific population of radio transients at $<100$ MHz. However, subsequent transient searches in similar parameter space yielded no detections. We test the hypothesis that these surveys at comparable sensitivity have missed the population due to mismatched survey parameters. In particular, the LOFAR survey used only $195$ kHz of bandwidth at $60$ MHz while other survey were at higher frequencies or had wider bandwidth. Using $137$ hours of all-sky images from the Owens Valley Radio…
The Australian Square Kilometre Array Pathfinder (ASKAP) collects images of the sky at radio wavelengths with an unprecedented field of view, combined with a high angular resolution and sub-millijansky sensitivities. The large quantity of data produced is used by the ASKAP Variables and Slow Transients (VAST) survey science project to study the dynamic radio sky. Efficient pipelines are vital in such research, where searches often form a `needle in a haystack' type of problem to solve. However, the existing pipelines developed among the radio-transient community are not suitable for the scale …
AST3-3 is the third robotic facility of the Antarctic Survey Telescopes (AST3) for transient surveys to be deployed at Dome A, Antarctica. Due to the current pandemic, the telescope has been currently deployed at the Yaoan Observation Station in China, starting the commissioning observation and a transient survey. This paper presents a fully automatic data processing system for AST3-3 observations. The transient detection pipeline uses state-of-the-art image subtraction techniques optimised for GPU devices. Image reduction and transient photometry are accelerated by concurrent task methods. Ou…
FIP-TOI: Fast Imaging Pipeline for Pulsar Localisation with a Transient-Oriented Radio Astronomical Imager
University of Oxford（牛津大学）；Silesian University in Opava（奥帕瓦西里西亚大学）；Mila - Québec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）；University of Cambridge（剑桥大学）；Special Astrophysical Observatory of the Russian Academy of Sciences（俄罗斯科学院特别天体物理观测站）
We present the Spectroscopic Classification of Astronomical Transients (SCAT) survey, which is dedicated to spectrophotometric observations of transient objects such as supernovae and tidal disruption events. SCAT uses the SuperNova Integral-Field Spectrograph (SNIFS) on the University of Hawai'i 2.2-meter (UH2.2m) telescope. SNIFS was designed specifically for accurate transient spectrophotometry, including absolute flux calibration and host-galaxy removal. We describe the data reduction and calibration pipeline including spectral extraction, telluric correction, atmospheric characterization,…
The best of all worlds: gravitational-wave transient detection with multiple pipelines
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；Institute for Accelerated AI Algorithms for Data-Driven Discovery (A3D3), MIT（麻省理工学院数据驱动发现加速AI算法研究所）；University of Thessaly（色萨利大学）
We present a methodology for automated real-time analysis of a radio image data stream with the goal to find transient sources. Contrary to previous works, the transients we are interested in occur on a time-scale where dispersion starts to play a role, so we must search a higher-dimensional data space and yet work fast enough to keep up with the data stream in real time. The approach consists of five main steps: quality control, source detection, association, flux measurement, and physical parameter inference. We present parallelized methods based on convolutions and filters that can be accel…
The scientific interest in studying high-energy transient phenomena in the Universe has largely grown for the last decade. Now, multiple ground-based survey projects have emerged to continuously monitor the optical (and multi-messenger) transient sky at higher image cadences and cover always larger portions of the sky every night. These novel approaches lead to a huge increase of the global alert rates which need to be handled with care especially by keeping the false alarms as low as possible. Therefore, the standard transient detection pipelines previously designed for narrow field of view i…
The Simons Observatory: Development of a Pipeline to Detect Rapid Transients in Time-Ordered Data
The University of Melbourne（墨尔本大学）；The International School for Advanced Studies (SISSA)（国际高等研究院）；The National Institute for Nuclear Physics (INFN)（意大利国家核物理研究所）；The National Institute for Astrophysics (INAF)（意大利国家天体物理研究所）；The Institute for Fundamental Physics of the Universe (IFPU)（宇宙基础物理研究所）；University of Pennsylvania（宾夕法尼亚大学）；University of Milano-Bicocca（米兰比可卡大学）；Princeton University（普林斯顿大学）；University of Toronto（多伦多大学）；Pontificia Universidad Católica de Chile（智利天主教 Pontificia 大学）；Specola Vaticana (Vatican Observatory)（梵蒂冈天文台）
$\textit{Context.}$ The magnetized Solar System planets are strong radio emitters and theoretical studies suggest that the radio emission from nearby exoplanets in close-in orbits could reach intensity levels $10^{3}-10^{7}$ times higher than Jupiter's decametric emission. Detection of exoplanets in the radio domain would open up a brand new field of research, however, currently there are no confirmed detections at radio frequencies. $\textit{Aims.}$ We investigate the radio emission from Jupiter, scaled such that it mimics emission coming from an exoplanet, with low-frequency beam-formed obse…
