Future generations of radio interferometers targeting the 21\,cm signal at cosmological distances with $N\gg 1000$ antennas could face a significant computational challenge in building correlators with the traditional architecture, whose computational resource requirement scales as $\mathcal{O}(N^2)$ with array size. The fundamental output of such correlators is the cross-correlation products of all antenna pairs in the array. The FFT-correlator architecture reduces the computational resources scaling to $\mathcal{O}(N\log{N})$ by computing cross-correlation products through a spatial Fourier transform. However, the output of the FFT-correlator is meaningful only when the input antenna voltages are gain- and phase-calibrated. Traditionally, interferometric calibration has used the $\mathcal{O}(N^2)$ cross-correlations produced by a standard correlator. This paper proposes two real-time calibration schemes that could work in parallel with an FFT-correlator as a self-contained $\mathcal{O}(N\log{N})$ correlator system that can be scaled to large-N redundant arrays. We compare the performance and scalability of these two calibration schemes and find that they result in antenna gains whose variance decreases as $1/\log{N}$ with increase in the size of the array.
Growing interest in 21 cm tomography has led to the design and construction of broadband radio interferometers with low noise, moderate angular resolution, high spectral resolution, and wide fields of view. With characteristics somewhat different from traditional radio instruments, these interferometers may require new calibration techniques in order to reach their design sensitivities. Self-calibration or redundant calibration techniques that allow an instrument to be calibrated off complicated sky emission structures are ideal. In particular, the large number of redundant baselines possessed…
We study the impact of sky-based calibration errors from source mismodeling on 21\,cm power spectrum measurements with an interferometer and propose a method for suppressing their effects. While emission from faint sources that are not accounted for in calibration catalogs is believed to be spectrally smooth, deviations of true visibilities from model visibilities are not, due to the inherent chromaticity of the interferometer's sky-response (the "wedge"). Thus, unmodeled foregrounds, below the confusion limit of many instruments, introduce frequency structure into gain solutions on the same l…
We use information theory to derive fundamental limits on the capacity to calibrate next-generation radio interferometers, and measure parameters of point sources for instrument calibration, point source subtraction, and data deconvolution. We demonstrate the implications of these fundamental limits, with particular reference to estimation of the 21cm Epoch of Reionization power spectrum with next-generation low-frequency instruments (e.g., the Murchison Widefield Array -- MWA, Precision Array for Probing the Epoch of Reionization -- PAPER), where short time scale instrumental calibration is r…
Redundant calibration is a technique in radio astronomy that allows calibration of radio arrays whose antennas lie on a lattice by exploiting the fact that redundant baselines should see the same sky signal. Because the number of measured visibilities scales quadratically with the number of antennas but the number of unknowns describing the individual antenna responses and the available information about the sky scales only linearly with the array size, the problem is always over-constrained as long as the array is big and dense enough. This is true even for non-lattice array configurations. I…
The calibration of modern radio interferometers is a significant challenge, specifically at low frequencies. In this perspective, we propose a novel iterative calibration algorithm, which employs the popular sparse representation framework, in the regime where the propagation conditions shift dissimilarly the directions of the sources. More precisely, our algorithm is designed to estimate the apparent directions of the calibration sources, their powers, the directional and undirectional complex gains of the array elements and their noise powers, with a reasonable computational complexity. Nume…
In order to meet the theoretically achievable imaging performance, calibration of modern radio interferometers is a mandatory challenge, especially at low frequencies. In this perspective, we propose a novel parallel iterative multi-wavelength calibration algorithm. The proposed algorithm estimates the apparent directions of the calibration sources, the directional and undirectional complex gains of the array elements and their noise powers, with a reasonable computational complexity. Furthermore, the algorithm takes into account the specific variation of the aforementioned parameter values ac…
High precison calibration is essential for a new generation of radio interferometers looking for Epoch of Reionization and Baryon Acoustic Oscillation signatures in neutral hydrogen. These arrays have so far been calibrated by redundant calibration, which usually assumes baselines intended to be identical are perfectly so. We present a new calibration scheme that relaxes the assumption of explicit redundancy by calculating the expected covariance of baselines. The technique also allows one to take advantage of partial knowledge of the sky, such as point sources with known positions but unknown…
In this paper, we investigate the calibration of radio interferometers in which Jones matrices are considered to model the interaction between the incident electromagnetic field and the antennas of each station. Specifically, perturbation effects are introduced along the signal path, leading to the conversion of the plane wave into an electric voltage by the receptor. In order to design a robust estimator, the noise is assumed to follow a spherically invariant random process (SIRP). The derived algorithm is based on an iterative relaxed concentrated maximum likelihood estimator (MLE), for whic…
In radio astronomy, accurate calibration is of crucial importance for the new generation of radio interferometers. More specifically, because of the potential presence of outliers which affect the measured data, robustness needs to be ensured. On the other hand, calibration is improved by taking advantage of these new instruments and exploiting the known structure of parameters of interest across frequency. Therefore, we propose in this paper an iterative robust multi-frequency calibration algorithm based on a distributed and consensus optimization scheme which aims to estimate the complex gai…
Having an accurate calibration method is crucial for any scientific research done by a radio telescope. The next generation radio telescopes such as the Square Kilometre Array (SKA) will have a large number of receivers which will produce exabytes of data per day. In this paper we propose new direction-dependent and independent calibration algorithms that, while requiring much less storage during calibration, converge very fast. The calibration problem can be formulated as a non-linear least square optimization problem. We show that combining a block-LDU decomposition with Gauss-Newton iterati…
Radio interferometers can measure the full polarization state of incoming waves by cross--correlating all combinations of two orthogonal polarizations at each antenna. The independent sets of electronics used to detect the two polarization states will introduce a differential instrumental delay between the two data streams. The usual technique of separate calibration of the parallel--hand sets of visibilities still allows for an arbitrary offset in group delay and phase between the two parallel systems. In order to use the cross--polarized visibilities, this instrumental offset must be determi…
Spectral features introduced by instrumental chromaticity of radio interferometers have the potential to negatively impact the ability to perform Epoch of Reionisation (EoR) and Cosmic Dawn (CD) science using the redshifted neutral hydrogen emission line from the early Universe. We describe instrument calibration choices that influence the spectral characteristics of the science data, and assess their impact on EoR statistical and tomographic experiments. Principally, we consider the intrinsic spectral response of the receiving antennas, embedded within a complete frequency-dependent primary b…
We introduce CALAMITY, a precision bandpass calibration method for radio interferometry. CALAMITY can solve for direction independent gains with arbitrary frequency structure to the high precision required for 21 cm cosmology with minimal knowledge of foregrounds or antenna beams and does not require any degree of redundancy (repeated identical measurements of the same baseline). We have achieved this through two key innovations. Firstly, we model the foregrounds on each baseline independently using a flexible and highly efficient set of basis functions that have minimal overlap with 21 cm mod…
Radio interferometer arrays with non-homogeneous element patterns are more difficult to calibrate compared to the more common homogeneous array. In particular, the non-homogeneity of the patterns has significant implications on the computational tractability of evaluating the calibration solutions. We apply the A-stacking technique to this problem and explore the trade-off to be made between the calibration accuracy and computational complexity. Through simulations, we show that this technique can be favourably applied in the context of an SKA-Low station. We show that the minimum accuracy req…
Paper I of the series re-derived the radio interferometry measurement equation (RIME) from first principles, and extended the Jones formalism to the full-sky case, incorporating direction-dependent effects (DDEs). This paper aims to describe both classical radio interferometric calibration (selfcal and related methods), and the recent developments in the treatment of DDEs, using the RIME-based mathematical framework developed in Paper I. It also aims to demonstrate the ease with which the various effects can be described and understood. The first section of this paper uses the RIME formalism t…
In this paper, we aim to design robust estimation techniques based on the compound-Gaussian (CG) process and adapted for calibration of radio interferometers. The motivation beyond this is due to the presence of outliers leading to an unrealistic traditional Gaussian noise assumption. Consequently, to achieve robustness, we adopt a maximum a posteriori (MAP) approach which exploits Bayesian statistics and follows a sequential updating procedure here. The proposed algorithm is applied in a multi-frequency scenario in order to enhance the estimation and correction of perturbation effects. Numeri…
Global 21-cm experiments require exquisitely precise calibration of the measurement systems in order to separate the weak 21-cm signal from Galactic and extragalactic foregrounds as well as instrumental systematics. Hitherto, experiments aiming to make this measurement have concentrated on measuring this signal using the single element approach. However, an alternative approach based on interferometers with short baselines is expected to alleviate some of the difficulties associated with a single element approach such as precision modelling of the receiver noise spectrum. Short spacing Interfe…
This manual is intended to help ALMA and other interferometer users improve images by recognising limitations and how to overcome them and deciding when and how to use self-calibration. The images provided by the ALMA Science Archive are calibrated using standard observing and data processing routines, including a quality assurance process to make sure that the observations meet the proposer's science requirements. This may not represent the full potential of the data, since any interferometry observation can be imaged with a range of resolutions and surface brightness sensitivity. The separat…
This paper investigates calibration of sensor arrays in the radio astronomy context. Current and future radio telescopes require computationally efficient algorithms to overcome the new technical challenges as large collecting area, wide field of view and huge data volume. Specifically, we study the calibration of radio interferometry stations with significant direction dependent distortions. We propose an iterative robust calibration algorithm based on a relaxed maximum likelihood estimator for a specific context: i) observations are affected by the presence of outliers and ii) parameters of …
Interferometry with phased-array feeds (PAFs) presents new calibration challenges in comparison with single-pixel feeds. In particular, temporal instability of the compound beam patterns due to element gain drifts (EGDs) can produce calibration artefacts in interferometric images. To translate imaging dynamic range requirements into PAF hardware and calibration requirements, we must learn to relate EGD levels to imaging artefact levels. We present a MeqTrees-based simulations framework that addresses this problem, and apply it to the APERTIF prototype currently in development for the WSRT.
We present a filtering technique that can be applied to individual baselines of wide-bandwidth, wide-field interferometric data to geometrically select regions on the celestial sphere that contain primary calibration sources. The technique relies on the Fourier transformation of wide-band frequency spectra from a given baseline to obtain one-dimensional "delay images", and then the transformation of a time-series of delay images to obtain two-dimensional "delay/delay-rate images." Source selection is possible in these images given appropriate combinations of baseline, bandwidth, integration ti…
The response of the Advanced LIGO interferometers is known to vary with time [arXiv:1608.05134]. Accurate calibration of the interferometers must therefore track and compensate for temporal variations in calibration model parameters. These variations were tracked during the first three Advanced LIGO observing runs, and compensation for some of them has been implemented in the calibration procedure. During the second observing run, multiplicative corrections to the interferometer response were applied while producing calibrated strain data both in real-time and in high-latency. In a high-latenc…
The Canadian Hydrogen Intensity Mapping Experiment (CHIME) is a transit interferometer currently being built at the Dominion Radio Astrophysical Observatory (DRAO) in Penticton, BC, Canada. We will use CHIME to map neutral hydrogen in the frequency range 400 -- 800\,MHz over half of the sky, producing a measurement of baryon acoustic oscillations (BAO) at redshifts between 0.8 -- 2.5 to probe dark energy. We have deployed a pathfinder version of CHIME that will yield constraints on the BAO power spectrum and provide a test-bed for our calibration scheme. I will discuss the CHIME calibration re…
We describe techniques concerning wavelength calibration and sky subtraction to maximise the scientific utility of data from tunable filter instruments. While we specifically address data from the Optical System for Imaging and low Resolution Integrated Spectroscopy instrument (OSIRIS) on the 10.4~m Gran Telescopio Canarias telescope, our discussion is generalisable to data from other tunable filter instruments. A key aspect of our methodology is a coordinate transformation to polar coordinates, which simplifies matters when the tunable filter data is circularly symmetric around the optical ce…
With ever increasing data rates produced by modern radio telescopes like LOFAR and future telescopes like the SKA, many data processing steps are overwhelmed by the amount of data that needs to be handled using limited compute resources. Calibration is one such operation that dominates the overall data processing computational cost, nonetheless, it is an essential operation to reach many science goals. Calibration algorithms do exist that scale well with the number of stations of an array and the number of directions being calibrated. However, the remaining bottleneck is the raw data volume, w…
From the Virgo interferometer calibration to the bias and uncertainty of the h(t) detector strain during the O4 run
Université de Strasbourg, CNRS, IPHC UMR 7178（斯特拉斯堡大学）；Laboratoire d’Annecy de Physique des Particules (LAPP), Univ. Grenoble Alpes, Université Savoie Mont Blanc, CNRS/IN2P3（阿讷西粒子物理实验室）；Institut de Fisica d’Altes Energies (IFAE), Barcelona Institute of Science and Technology, and ICREA（高能物理研究所）；Universiteit Antwerpen（安特卫普大学）
Gravitational-wave (GW) laser interferometers such as Advanced LIGO transduce spacetime strain into optical power fluctuation. Converting this optical power fluctuations back into an estimated spacetime strain requires a calibration process that accounts for both the interferometer's optomechanical response and the feedback control loop used to control the interferometer test masses. Systematic errors in the calibration parameters lead to systematic errors in the GW strain estimate, and hence to systematic errors in the astrophysical parameter estimates in a particular GW signal. In this work …
The formulation of the radio interferometer measurement equation (RIME) by Hamaker et al. has provided us with an elegant mathematical apparatus for better understanding, simulation and calibration of existing and future instruments. The calibration of the new radio telescopes (LOFAR, SKA) would be unthinkable without the RIME formalism, and new software to exploit it. MeqTrees is designed to implement numerical models such as the RIME, and to solve for arbitrary subsets of their parameters. The technical goal of MeqTrees is to provide a tool for rapid implementation of such models, while offe…
The 21-cm signal of neutral hydrogen - emitted during the Epoch of Reionization - promises to be an important source of information for the study of the infant universe. However, its detection is impossible without sufficient mitigation of other strong signals in the data, which requires an accurate knowledge of the instrument. Using the result of instrument calibration, a large part of the contaminating signals are removed and the resulting residual data is further analyzed in order to detect the 21-cm signal. Direction dependent calibration (DDC) can strongly affect the 21-cm signal, however…
Papers I and II of this series have extended the radio interferometry measurement equation (RIME) formalism to the full-sky case, and provided a RIME-based description of calibration and the problem of direction-dependent effects (DDEs). This paper aims to provide a practical demonstration of a RIME-based approach to calibration, via an example of extremely high-dynamic range calibration of WSRT observations of 3C 147 at 21 cm, with full treatment of DDEs. A version of the RIME incorporating differential gains has been implemented in MeqTrees, and applied to the 3C 147 data. This was used to p…
For a ground-based radio interferometer observing at low frequencies, the ionosphere causes propagation delays and refraction of cosmic radio waves which result in phase errors in the received signal. These phase errors can be corrected using a calibration method that assumes a two-dimensional phase screen at a fixed altitude above the surface of the Earth, known as the thin-layer model. Here we investigate the validity of the thin-layer model and provide a simple equation with which users can check when this approximation can be applied to observations for varying time of day, zenith angle, i…
We discuss the ongoing development of single-mode fiber Fabry-Perot (FFP) Interferometers as precise astro-photonic calibration sources for high precision radial velocity (RV) spectrographs. FFPs are simple, inexpensive, monolithic units that can yield a stable and repeatable output spectrum. An FFP is a unique alternative to a traditional etalon, as the interferometric cavity is made of single-mode fiber rather than an air-gap spacer. This design allows for excellent collimation, high spectral finesse, rigid mechanical stability, insensitivity to vibrations, and no need for vacuum operation. …
We report on the MIT Epoch of Reionization (MITEoR) experiment, a pathfinder low-frequency radio interferometer whose goal is to test technologies that improve the calibration precision and reduce the cost of the high-sensitivity 3D mapping required for 21 cm cosmology. MITEoR accomplishes this by using massive baseline redundancy, which enables both automated precision calibration and correlator cost reduction. We demonstrate and quantify the power and robustness of redundancy for scalability and precision. We find that the calibration parameters precisely describe the effect of the instrumen…
Since its formulation by Hamaker et al., the radio interferometer measurement equation (RIME) has provided a rigorous mathematical basis for the development of novel calibration methods and techniques, including various approaches to the problem of direction-dependent effects (DDEs). This series of papers aims to place recent developments in the treatment of DDEs into one RIME-based mathematical framework, and to demonstrate the ease with which the various effects can be described and understood. It also aims to show the benefits of a RIME-based approach to calibration. Paper I re-derives the …
In the past two decades, a rebirth of interest in low-frequency radio astronomy for 21 cm tomography of the Epoch of Reionization, has given rise to a new class of radio interferometers with $N \gg 100$ antennas. The availability of low-noise receivers that do not require cryogenic cooling has driven down the cost of antennas, making it affordable to build sensitivity with numerous small antennas rather than large dish structures. However, the computational- and storage-costs of such radio arrays, determined by the $\mathcal{O}(N^2)$ scaling of visibility products required for calibration and …
AMIGO: a Data-Driven Calibration of the JWST Interferometer
Sydney Institute for Astronomy, School of Physics, University of Sydney（悉尼大学物理学院天体物理研究所）；School of Mathematical & Physical Sciences, Macquarie University（麦考瑞大学数学与物理科学学院）；Department of Physics and Astronomy, University of Victoria（维多利亚大学物理与天文系）；Space Telescope Science Institute（太空望远镜科学研究所）；European Southern Observatory（欧洲南方天文台）
本文提出AMIGO框架，通过端到端可微架构校准JWST干涉仪，提升高对比度成像性能，成功恢复AB Dor AC双星并发现HD206893B和HD206893c，超越现有算法。
In the last decade, white-light illuminated Fabry-Pérot interferometers wave been established as a widely used, relatively simple, reliable, and cost-effective way to precisely calibrate high-resolution echelle spectrographs. However, Terrien et al. (2021) recently reported a chromatic drift of the Fabry-Pérot interferometer installed at the Habitable-zone Planet Finder spectrograph. In particular, they found that the variation of the etalon effective gap size is not achromatic as usually assumed but in fact depends on wavelength. Here, we present a similar study of the Espresso Fabry-Pérot in…
Our ability to calibrate current kilometer-scale interferometers can potentially confound the inference of astrophysical signals. Current calibration uncertainties are well described by a Gaussian process. I exploit this description to analytically examine the impact of calibration uncertainty. I derive closed-form expressions for the conditioned likelihood of the calibration error given the observed data and an astrophysical signal (astrophysical calibration) as well as for the marginal likelihood for the data given a signal (integrated over the calibration uncertainty). I show that calibrati…
The Parkes-Tidbinbilla took advantage of a real-time radio-link connecting the Parkes and Tidbinbilla antennas to form the world's longest real-time interferometer, perhaps the earliest example of eVLBI. Built on a minuscule budget, it was an extraordinarily successful instrument, generating some 24 journal papers including 3 Nature papers, as well as facilitating the early development of the Australia Telescope Compact Array. Here we describe its origins, construction, successes, and life cycle, and discuss the future use of single-baseline interferometers in the era of SKA and its pathfinder…
In order to cope with the increased data volumes generated by modern radio interferometers such as LOFAR (Low Frequency Array) or SKA (Square Kilometre Array), fast and efficient calibration algorithms are essential. Traditional radio interferometric calibration is performed using nonlinear optimization techniques such as the Levenberg-Marquardt algorithm in Euclidean space. In this paper, we reformulate radio interferometric calibration as a nonlinear optimization problem on a Riemannian manifold. The reformulated calibration problem is solved using the Riemannian trust-region method. We show…
While most purpose-built 21cm intensity mapping experiments are close-packed interferometer arrays, general-purpose dish arrays should also be capable of measuring the cosmological 21cm signal. This can be achieved most efficiently if the array is used as a collection of scanning autocorrelation dishes rather than as an interferometer. As a first step towards demonstrating the feasibility of this observing strategy, we show that we are able to successfully calibrate dual-polarisation autocorrelation data from 64 MeerKAT dishes in the L-band (856-1712 MHz, 4096 channels), with 10.5 hours of dat…
Array configurations built on a hierarchy of simple elements have excellent properties for cross-correlation imaging interferometers including a smooth distribution of measured Fourier components, high angular resolution, low side lobes, and compact array size. Compared to arrays with a Gaussian distribution of antenna separations, hierarchical arrays (H-arrays) produce beams with higher angular resolution and a tighter concentration of the total power (encircled energy) within a smaller area around the main beam. An attractive feature of H-arrays is their simplicity. The relationships between…
The data reduction procedure for radio interferometers can be viewed as a combined calibration and imaging problem. We present an algorithm that unifies cross-calibration, self-calibration, and imaging. Being a Bayesian method, that algorithm does not only calculate an estimate of the sky brightness distribution, but also provides an estimate of the joint uncertainty which entails both the uncertainty of the calibration and the one of the actual observation. The algorithm is formulated in the language of information field theory and uses Metric Gaussian Variational Inference (MGVI) as the unde…
Context. Frequent, simultaneous observations across the electromagnetic spectrum are essential to the study of a range of astrophysical phenomena including Active Galactic Nuclei. A key tool of such studies is the ability to observe an object when it flares i.e. exhibits a rapid and significant increase in its flux density. Aims. We describe the specific observational procedures and the calibration techniques that have been developed and tested to create a single baseline radio interferometer that can rapidly observe a flaring object. This is the only facility that is dedicated to rapid high r…
The Parkes-Tidbinbilla took advantage of a real-time radio-link connecting the Parkes and Tidbinbilla antennas to form the world's longest real-time interferometer. Built on a minuscule budget, it was an extraordinarily successful instrument, generating some 24 journal papers including 3 Nature papers, as well as facilitating the early development of the Australia Telescope Compact Array. Here we describe its origins, construction, successes, and life cycle, and discuss the future use of single-baseline interferometers in the era of SKA and its pathfinders.
Large radio astronomy multi-element interferometers are frequently used as single dishes in a tied-array mode when signals from separate antennas are added. Phase shifts arising during wave propagation through a turbulent atmosphere can significantly reduce the effective area of an equivalent single dish. I aim to give estimates of the impact of the ionosphere and troposphere on the effectiveness of a radio interferometer working in tied-array mode. Statistical estimates of the effective area are calculated and the power-law of turbulent atmosphere irregularities has been used. A simple method…
As the sensitivities of LIGO, Virgo and KAGRA detectors improve, calibration of the interferometers output is becoming more and more important and may impact scientific results. For the observing run O3, Virgo used for the first time photon calibrators (PCal) to calibrate the interferometer, using radiation pressure of a modulated auxiliary laser beam impinging on the Advanced Virgo end mirrors. Those optical devices, also used in LIGO, are now the calibration reference for the global gravitational wave detectors network. The intercalibration of LIGO and Virgo PCals, based on the same absolute…
Distributed calibration based on consensus optimization is a computationally efficient method to calibrate large radio interferometers such as LOFAR and SKA. Calibrating along multiple directions in the sky and removing the bright foreground signal is a crucial step in many science cases in radio interferometry. The residual data contain weak signals of huge scientific interest and of particular concern is the effect of incomplete sky models used in calibration on the residual. In order to study this, we consider the mapping between the input uncalibrated data and the output residual data. We …
HERA is a highly-redundant transit interferometer with 14\,m-diameter parabolic dish elements. We exploit the fact that the Galactic centre transits through the main beam of the telescope to attempt a conventional self-calibration approach to imaging and calibration. The Galactic centre provides a bright source which, we show, can be approximated as a point source sufficiently well to initialise the self-calibration loop and derive initial delays and antenna frequency-independent phases. Subsequent iteration using a more complex sky model derived from the data itself then converges to a reason…
We have built a reliable and robust system that takes as input an astronomical image, and returns as output the pointing, scale, and orientation of that image (the astrometric calibration or WCS information). The system requires no first guess, and works with the information in the image pixels alone; that is, the problem is a generalization of the "lost in space" problem in which nothing--not even the image scale--is known. After robust source detection is performed in the input image, asterisms (sets of four or five stars) are geometrically hashed and compared to pre-indexed hashes to genera…
Increasing data volumes delivered by a new generation of radio interferometers require computationally efficient and robust calibration algorithms. In this paper, we propose distributed calibration as a way of improving both computational cost as well as robustness in calibration. We exploit the data parallelism across frequency that is inherent in radio astronomical observations that are recorded as multiple channels at different frequencies. Moreover, we also exploit the smoothness of the variation of calibration parameters across frequency. Data parallelism enables us to distribute the comp…
