The aim of the new generation of radio synthesis arrays such as LOFAR and SKA is to achieve much higher sensitivity, resolution and frequency coverage than what is available now, especially at low frequencies. To accomplish this goal, the accuracy of the calibration techniques used is of considerable importance. Moreover, since these telescopes produce huge amounts of data, speed of convergence of calibration is a major bottleneck. The errors in calibration are due to system noise (sky and instrumental) as well as the estimation errors introduced by the calibration technique itself, which we call solver noise. We define solver noise as the distance between the optimal solution (the true value of the unknowns, uncorrupted by the system noise) and the solution obtained by calibration. We present the Space Alternating Generalized Expectation Maximization (SAGE) calibration technique, which is a modification of the Expectation Maximization algorithm, and compare its performance with the traditional Least Squares calibration based on the level of solver noise introduced by each technique. For this purpose, we develop statistical methods that use the calibrated solutions to estimate the level of solver noise. The SAGE calibration algorithm yields very promising results both in terms of accuracy and speed of convergence. The comparison approaches we adopt introduce a new framework for assessing the performance of different calibration schemes.
Radio interferometric gain calibration can be biased by incomplete sky models and radio frequency interference, resulting in calibration artefacts that can restrict the dynamic range of the resulting images. It has been suggested that calibration algorithms employing heavy-tailed likelihood functions are less susceptible to this due to their robustness against outliers in the data. We present an algorithm based on a Student's t-distribution which leverages the framework of complex optimisation and Wirtinger calculus for efficient and robust interferometric gain calibration. We integrate this a…
Observations of the redshifted 21-cm line from the epoch of reionization have recently motivated the construction of low frequency radio arrays with highly redundant configurations. These configurations provide an alternative calibration strategy - "redundant calibration" - and boosts sensitivity on specific spatial scales. In this paper, we formulate calibration of redundant interferometric arrays as a complex optimization problem. We solve this optimization problem via the Levenberg-Marquardt algorithm. This calibration approach is more robust to initial conditions than current algorithms an…
The data reduction procedure for radio interferometers can be viewed as a combined calibration and imaging problem. We present an algorithm that unifies cross-calibration, self-calibration, and imaging. Being a Bayesian method, that algorithm does not only calculate an estimate of the sky brightness distribution, but also provides an estimate of the joint uncertainty which entails both the uncertainty of the calibration and the one of the actual observation. The algorithm is formulated in the language of information field theory and uses Metric Gaussian Variational Inference (MGVI) as the unde…
In this paper we present DMC, a model and associated tool for polarimetric imaging of very long baseline interferometry datasets that simultaneously reconstructs the full-Stokes emission structure along with the station-based gain and leakage calibration terms. DMC formulates the imaging problem in terms of posterior exploration, which is achieved using Hamiltonian Monte Carlo sampling. The resulting posterior distribution provides a natural quantification of uncertainty in both the image structure and in the data calibration. We run DMC on both synthetic and real datasets, the results of whic…
It has recently been shown that radio interferometric gain calibration can be expressed succinctly in the language of complex optimisation. In addition to providing an elegant framework for further development, it exposes properties of the calibration problem which can be exploited to accelerate traditional non-linear least squares solvers such as Gauss-Newton and Levenberg-Marquardt. We extend existing derivations to chains of Jones terms: products of several gains which model different aberrant effects. In doing so, we find that the useful properties found in the single term case still hold.…
Solution intervals are often used to improve the signal-to-noise ratio during radio interferometric gain calibration. This work investigates how factors such as the noise level, intrinsic gain variability, degree of model incompleteness, and the presence of radio frequency interference impact the selection of solution intervals for calibration. We perform different interferometric simulations to demonstrate how these factors, in combination with the choice of solution intervals, affect calibration and imaging outputs and discuss practical guidelines for choosing optimal solution intervals. Fur…
Calibration is a key step in the signal processing pipeline of any radio astronomical instrument. The required sky, ionospheric and instrumental models for this step can suffer from various kinds of incompleteness. In this paper we analyze several important calibration methods, ignoring for now the ionosphere. The aim is to use established statistical and signal processing tools to provide a generic method to assess calibratability of an instrument. We show how currently popular calibration techniques differ in their assumptions and also discuss their theoretical commonalities. We also study t…
Calibration of a typical radio interferometric array yields thousands of parameters as solutions. These solutions contain valuable information about the systematic errors in the data (ionosphere and beam shape). This information could be reused in calibration to improve the accuracy and also can be fed into imaging to improve the fidelity. We propose a distributed optimization strategy to construct models for the systematic errors in the data using the calibration solutions. We formulate this as an elastic net regularized distributed optimization problem which we solve using the alternating di…
Interferometric arrays seeking to measure the 21 cm signal from the Epoch of Reionization must contend with overwhelmingly bright emission from foreground sources. Accurate recovery of the 21 cm signal will require precise calibration of the array, and several new avenues for calibration have been pursued in recent years, including methods using redundancy in the antenna configuration. The newly upgraded Phase II of Murchison Widefield Array (MWA) is the first interferometer that has large numbers of redundant baselines while retaining good instantaneous UV-coverage. This array therefore provi…
Correlation Calibration: A Hybrid Calibration Technique for Radio Interferometric Arrays
McGill University（麦吉尔大学）；Dunlap Institute for Astronomy and Astrophysics, University of Toronto（多伦多大学邓普勒天文学与天体物理研究所）；Perimeter Institute for Theoretical Physics（基础物理研究所）
High fidelity radio interferometric data calibration that minimises spurious spectral structure in the calibrated data is essential in astrophysical applications, such as 21 cm cosmology, which rely on knowledge of the relative spectral smoothness of distinct astrophysical emission components to extract the signal of interest. Existing approaches to radio interferometric calibration have been shown to impart spurious spectral structure to the calibrated data if the sky model used to calibrate the data is incomplete. In this paper, we introduce BayesCal: a novel solution to the sky-model incomp…
Distributed calibration based on consensus optimization is a computationally efficient method to calibrate large radio interferometers such as LOFAR and SKA. Calibrating along multiple directions in the sky and removing the bright foreground signal is a crucial step in many science cases in radio interferometry. The residual data contain weak signals of huge scientific interest and of particular concern is the effect of incomplete sky models used in calibration on the residual. In order to study this, we consider the mapping between the input uncalibrated data and the output residual data. We …
This paper investigates the possibility of improving radio interferometric images using an algorithm inspired by an optical method known as "lucky imaging", which would give more weight to the best-calibrated visibilities used to make a given image. A fundamental relationship between the statistics of interferometric calibration solution residuals and those of the image-plane pixels is derived in this paper. This relationship allows us to understand and describe the statistical properties of the residual image. In this framework, the noise-map can be described as the Fourier transform of the c…
In a companion paper, we presented BayesCal, a mathematical formalism for mitigating sky-model incompleteness in interferometric calibration. In this paper, we demonstrate the use of BayesCal to calibrate the degenerate gain parameters of full-Stokes simulated observations with a HERA-like hexagonal close-packed redundant array, for three assumed levels of completeness of the a priori known component of the calibration sky model. We compare the BayesCal calibration solutions to those recovered by calibrating the degenerate gain parameters with only the a priori known component of the calibrati…
Interferometric imaging now achieves angular resolutions as fine as 10 microarcsec, probing scales that are inaccessible to single telescopes. Traditional synthesis imaging methods require calibrated visibilities; however, interferometric calibration is challenging, especially at high frequencies. Nevertheless, most studies present only a single image of their data after a process of "self-calibration," an iterative procedure where the initial image and calibration assumptions can significantly influence the final image. We present a method for efficient interferometric imaging directly using …
Calibration is an essential step in radio interferometric data processing that corrects the data for systematic errors and in addition, subtracts bright foreground interference to reveal weak signals hidden in the residual. These weak and unknown signals are much sought after to reach many science goals but the effect of calibration on such signals is an ever present concern. The main reason for this is the incompleteness of the model used in calibration. Distributed calibration based on consensus optimization has been shown to mitigate the effect due to model incompleteness by calibrating dat…
A Formalism for Calibrating the Instrumental Polarization of Radio Interferometric Arrays at Meter Wavelengths using Unpolarized Sky: A Demonstration using the MWA Observations
ASTRON, The Netherlands Institute for Radio Astronomy（荷兰射电天文研究所（ASTRON））；Netherlands eScience Center（荷兰电子科学中心）
We present a stochastic, limited-memory Broyden Fletcher Goldfarb Shanno (LBFGS) algorithm that is suitable for handling very large amounts of data. A direct application of this algorithm is radio interferometric calibration of raw data at fine time and frequency resolution. Almost all existing radio interferometric calibration algorithms assume that it is possible to fit the dataset being calibrated into memory. Therefore, the raw data is averaged in time and frequency to reduce its size by many orders of magnitude before calibration is performed. However, this averaging is detrimental for th…
The celebrated CLEAN algorithm has been the cornerstone of deconvolution algorithms in radio interferometry almost since its conception in the 1970s. For all its faults, CLEAN is remarkably fast, robust to calibration artefacts and in its ability to model point sources. We demonstrate how the same assumptions that afford CLEAN its speed can be used to accelerate more sophisticated deconvolution algorithms.
Interferometric visibilities, reflecting the complex correlations between signals recorded at antennas in an interferometric array, carry information about the angular structure of a distant source. While unknown antenna gains in both amplitude and phase can prevent direct interpretation of these measurements, certain combinations of visibilities called closure phases and closure amplitudes are independent of antenna gains and provide a convenient set of robust observables. However, these closure quantities have subtle noise properties and are generally both linearly and statistically dependen…
New and upcoming radio interferometers will produce unprecedented amounts of data that demand extremely powerful computers for processing. This is a limiting factor due to the large computational power and energy costs involved. Such limitations restrict several key data processing steps in radio interferometry. One such step is calibration where systematic errors in the data are determined and corrected. Accurate calibration is an essential component in reaching many scientific goals in radio astronomy and the use of consensus optimization that exploits the continuity of systematic errors acr…
This chapter aims to provide a review of the basics of 21 cm interferometric observations and its methodologies. A summary of the main concepts of radio interferometry and their connection with the 21 cm observables - power spectra and images - is presented. I then provide a review of interferometric calibration and its interplay with foreground separation, including the current open challenges in calibration of 21 cm observations. Finally, a review of 21 cm instrument designs in the light of calibration choices and observing strategies follows.
Photometric calibration of the Stellar Abundance and Galactic Evolution Survey (SAGES): Nanshan One-meter Wide-field Telescope g, r, and i band imaging data
The ionosphere is the main driver of a series of systematic effects that limit our ability to explore the low frequency (<1 GHz) sky with radio interferometers. Its effects become increasingly important towards lower frequencies and are particularly hard to calibrate in the low signal-to-noise ratio regime in which low-frequency telescopes operate. In this paper we characterize and quantify the effect of ionospheric-induced systematic errors on astronomical interferometric radio observations at ultra-low frequencies (<100 MHz). We also provide guidelines for observations and data reduction at …
Radio Interferometric Calibration with the Exponential Map
Argentine Institute of Radio astronomy (IAR), UNLP - CONICET - CICPBA（阿根廷射电天文研究所（IAR），拉普拉塔国立大学-国家科学研究委员会-圣克鲁斯省科学和技术研究委员会）；LEICI Institute of Research in Electronics, Control and Signal Processing, UNLP - CONICET（电子、控制和信号处理研究所（LEICI），拉普拉塔国立大学-国家科学研究委员会）
In this lecture, we describe a number of advanced gain calibration techniques. In particular, self-calibration is an important tool in interferometric imaging at all wavelengths. It allows the observer to determine and remove residual phase and amplitude errors that remain in the data after normal calibration while simultaneously producing a more accurate and more sensitive image of the target source. We describe the basic technique of self-calibration and attempt to dispel some common misconceptions. We proceed to give a range of useful tips, and provide continuum, spectral line, and mosaic s…
The Primordial Inflation Explorer (PIXIE) is an Explorer-class mission concept to measure the gravitational-wave signature of primordial inflation through its distinctive imprint on the linear polarization of the cosmic microwave background (CMB). Its optical system couples a polarizing Fourier transform spectrometer to the sky to measure the differential signal between orthogonal linear polarization states from two co-pointed beams on the sky. The double differential nature of the four-port measurement mitigates beam-related systematic errors common to the two-port systems used in most CMB me…
Lossless compression of simulated radio interferometric visibilities
ASTRON, the Netherlands Institute for Radio Astronomy（荷兰射电天文研究所）；Kapteyn Astronomical Institute, University of Groningen（格罗宁根大学卡普坦天文研究所）；Leiden Observatory, Leiden University（莱顿大学莱顿天文台）
A Python-Based Peeling Framework for Radio Interferometry: Application to uGMRT 650MHz Imaging
Purple Mountain Observatory, Chinese Academy of Sciences（中国科学院紫金山天文台）；University of Science and Technology of China（中国科学技术大学）；Yunnan Observatories, Chinese Academy of Sciences（中国科学院云南天文台）；Nanjing University（南京大学）；National Radio Astronomy Observatory, NRAO（美国国家射电天文台）；University of Cape Town（开普敦大学）；University of the Western Cape（西开普大学）
A New Method for Aperture Masking Interferometric Imaging: Demonstration with the JWST
National Radio Astronomy Observatory（国家射电天文台）；Astrophysics Group, Cavendish Laboratory, Univ. of Cambridge（剑桥大学卡文迪许实验室天体物理组）；ALBA - CELLS Synchrotron Radiation Facility（阿尔巴-CELLS同步辐射设施）；Commonwealth Scientific and Industrial Research Organisation (CSIRO), Space & Astronomy（联邦科学与工业研究组织（CSIRO）空间与天文部门）
