Power Monitoring and Control for Large Scale projects: SKA, a case study
Domingos Barbosa, João Paulo Barraca, Dalmiro Maia, Bruno Carvalho, Jorge Vieira, Paul Swart, Gerhard Le Roux, Swaminathan Natarajan, Arnold van Ardenne, Luis Seca
备注：
11 pages, 3 figures, 3 tables, in press Proceedings of SPIE Astronomical Telescopes + Instrumentation 2016 (9910-25). arXiv admin note: text overlap with arXiv:1402.2783
Large sensor-based science infrastructures for radio astronomy like the SKA will be among the most intensive data-driven projects in the world, facing very high demanding computation, storage, management, and above all power demands. The geographically wide distribution of the SKA and and its associated processing requirements in the form of tailored High Performance Computing (HPC) facilities, require a Greener approach towards the Information and Communications Technologies (ICT) adopted for the data processing to enable operational compliance to potentially strict power budgets. Addressing the reduction of electricity costs, improve system power monitoring and the generation and management of electricity at system level is paramount to avoid future inefficiencies and higher costs and enable fulfillments of Key Science Cases. Here we outline major characteristics and innovation approaches to address power efficiency and long-term power sustainability for radio astronomy projects, focusing on Green ICT for science and Smart power monitoring and control.
SKA (Square Kilometer Array), currently under design, will be a huge radio-astronomical facility, whose management will be performed by a suite of software applications called Telescope Manager (SKA TM) via the TANGO framework. In order to ensure the proper and uninterrupted operation of TM, a local monitoring and control system (TM.LMC) is being developed, with the goal to perform monitoring, lifecycle control and fault management of TM. For the monitoring activity, central in TM.LMC, Nagios (automated by the lifecycle management tool Chef) has been proposed as main toolkit to check resources…
The concept of a Square Kilometre Array was developed to ensure that progress in Radio Astronomy in the early 21st Century continued at the same impressive pace as was achieved during the first 50 years. The SKA telescope is designed to pave that road to greater and greater sensitivity. So what technical challenges does the project face and what key innovations will drive the success of the SKA? What will the next Radio Astronomy mega-science project look like? In this article the author discusses the likely avenues of progress in the coming decades and comments on the status of radio astronom…
The SKA will be capable of producing a stream of science data products that are Exa-scale in terms of their storage and processing requirements. This Google-scale enterprise is attracting considerable international interest and excitement from within the industrial and academic communities. In this chapter we examine the data flow, storage and processing requirements of a number of key SKA survey science projects to be executed on the baseline SKA1 configuration. Based on a set of conservative assumptions about trends for HPC and storage costs, and the data flow process within the SKA Observat…
A significant fraction of the observing time with the two phase-I SKA telescopes (SKA1-LOW and SKA1-MID) will be spent on Key Science Projects led by member country scientists. The various SKA Science Working Groups, including the VLBI Focus Group are in the process of defining KSPs that are aligned with the High Priority Science Objectives of the SKA. At the moment it is not clear how the special observing mode of SKA-VLBI - when the SKA1 components are phased-up and included in VLBI networks - could be incorporated in KSPs. The VLBI community needs to be prepared by the time the KSP proposal…
This document describes the top level requirements for the SKA-AAMID telescope as determined by the SKA key science projects. These include parameters such as operating frequency range,instantaneous bandwidth (total processed bandwidth), field of view (or survey speed, as appropriate), sensitivity, dynamic range, polarization purity etc. Moreover, through the definition of a set of science requirements, this document serves as input to a number of other documents contained within the System Requirements Review package. (particularly SKA-TEL-MFAA-0200005: `SKA-AAMID System Requirements' and SKA…
In order to deliver the full science potential of the Square Kilometer Array (SKA) telescope, several SKA Regional Centres (SRCs) will be required to be constructed in different SKA member countries around the world. These SRCs will provide high performance compute and storage for the generation of advanced science data products from the basic data streams generated by the SKA Science Data Handling and Processing system, critically necessary to the success of the key science projects to be carried out by the SKA user community. They will also provide support to astronomers to enable them to ca…
New developments in data processing and visualization are being made in preparation for upcoming radioastronomical surveys planned with the Square Kilometre Array (SKA) and its precursors. A major goal is enabling extraction of science information from the data in a mostly automated way, possibly exploiting the capabilities offered by modern computing infrastructures and technologies. In this context, the integration of source analysis algorithms into data visualization tools is expected to significantly improve and speed up the cataloguing process of large area surveys. To this aim, the CIRAS…
We present the results of our investigations into options for the computing platform for the imaging pipeline in the CHILES project, an ultra-deep HI pathfinder for the era of the Square Kilometre Array. CHILES pushes the current computing infrastructure to its limits and understanding how to deliver the images from this project is clarifying the Science Data Processing requirements for the SKA. We have tested three platforms: a moderately sized cluster, a massive High Performance Computing (HPC) system, and the Amazon Web Services (AWS) cloud computing platform. We have used well-established …
Systems Engineering (SE) is the set of processes and documentation required for successfully realising large-scale engineering projects, but the classical approach is not a good fit for software-intensive projects, especially when the needs of the different stakeholders are not fully known from the beginning, and requirement priorities might change. The SKA is the ultimate software-enabled telescope, with enormous amounts of computing hardware and software required to perform its data reduction. We give an overview of the system and software engineering processes in the SKA1 development, and t…
The Square Kilometre Array (SKA) project consists of delivering two largest radio telescope arrays being built by the SKA Observatory (SKAO), which is an intergovernmental organization bringing together nations from around the world with China being one of the major member countries. The computing resources needed to process, distribute, curate and use the vast amount of data that will be generated by the SKA telescopes are too large for the SKAO to manage on its own. To address this challenge, the SKAO is working with the international community to create a shared, distributed data, computing…
A revolution in radio receiving technology is underway with the development of densely packed phased arrays for radio astronomy. This technology can provide an exceptionally large field of view, while at the same time sampling the sky with high angular resolution. Such an instrument, with a field of view of over 100 square degrees, is ideal for performing fast, all-sky, surveys, such as the "intensity mapping" experiment to measure the signature of Baryonic Acoustic Oscillations in the HI mass distribution at cosmological redshifts. The SKA, built with this technology, will be able to do a bil…
The Square Kilometre Array (SKA) project is an international cooperation project to build the largest radio telescope worldwide. Data processing is one of the biggest challenges of building the SKA telescope. As a distributed execution framework, the Data Activated Liu Graph Engine (DALiuGE) was proposed to be one of the candidates for addressing the massive data of the SKA. DALiuGE has many distinctive features, but its actual ability to handle scientific data is still not evident. In this paper, we perform an objective evaluation of the usability of DALiuGE concerning the execution performan…
"High-resolution", or "long-baseline", science with the SKA and its precursors covers a broad range of topics in astrophysics. In several research areas, the coupling between improved brightness sensitivity of the SKA and a sub-arcsecond resolution would uncover truly unique avenues and opportunities for studying extreme states of matter, vicinity of compact relativistic objects, and complex processes in astrophysical plasmas. At the same time, long baselines would secure excellent positional and astrometric measurements with the SKA and critically enhance SKA image fidelity at all scales. The…
Radio Astronomy is undergoing a renaissance, as the next-generation of instruments provides a massive leap forward in collecting area and therefore raw sensitivity. However, to achieve this theoretical level of sensitivity in the science data products we need to address the much more pernicious systematic effects, which are the true limitation. These become all the more significant when we consider that much of the time used by survey instruments, such as the SKA, will be dedicated to deep surveys. CHILES is a deep HI survey of the COSMOS field, with 1,000 hours of VLA time. We present our app…
Preliminary specifications for the Square Kilometre Array (SKA) call for 25% of the total collecting area of the dish array to be located at distances greater than 180 km from the core, with a maximum baseline of at least 3000 km. The array will provide angular resolution ~ 40 - 2 mas at 0.5 - 10 GHz with image sensitivity reaching < 50 nJy/beam in an 8 hour integration with 500 MHz bandwidth. Given these specifications, the high angular resolution component of the SKA will be capable of detecting brightness temperatures < 200 K with milliarcsecond-scale angular resolution. The aim of this art…
The Square Kilometre Array (SKA) will be both the largest radio telescope ever constructed and the largest Big Data project in the known Universe. The first phase of the project will generate on the order of 5 zettabytes of data per year. A critical task for the SKA will be its ability to process data for science, which will need to be conducted by science pipelines. Together with polarization data from the LOFAR Multifrequency Snapshot Sky Survey (MSSS), we have been developing a realistic SKA-like science pipeline that can handle the large data volumes generated by LOFAR at 150 MHz. The pipe…
Magnetic fields play essential roles in various astronomical objects. Radio astronomy has revealed that magnetic fields are ubiquitous in our Universe. However, the real origin and evolution of magnetic fields is poorly proven. In order to advance our knowledge of cosmic magnetism in coming decades, the Square Kilometre Array (SKA) should have supreme sensitivity than ever before, which provides numerous observation points in the cosmic space. Furthermore, the SKA should be designed to facilitate wideband polarimetry so as to allow us to examine sightline structures of magnetic fields by means…
I consider the question posed to me by the scientific organisers of the conference, "Are the modern computer simulations a substitute for physical models? The SKA case." I briefly consider the current knowledge of computer simulations and of physical prototypes in the context of understanding interferometric radio telescopes. My conclusion is that, "no, computer simulations are not a substitute for physical models when it comes to understanding the SKA.....furthermore, physical models are not much help either." This conclusion is intentionally provocative, designed to promote some discussion a…
On a time scale of years to decades, gravitational wave (GW) astronomy will become a reality. Low frequency (nanoHz) GWs are detectable through long-term timing observations of the most stable pulsars. Radio observatories worldwide are currently carrying out observing programmes to detect GWs, with data sets being shared through the International Pulsar Timing Array project. One of the most likely sources of low frequency GWs are supermassive black hole binaries (SMBHBs), detectable as a background due to a large number of binaries, or as continuous or burst emission from individual sources. N…
The Australian Square Kilometre Array Pathfinder (ASKAP) collects images of the sky at radio wavelengths with an unprecedented field of view, combined with a high angular resolution and sub-millijansky sensitivities. The large quantity of data produced is used by the ASKAP Variables and Slow Transients (VAST) survey science project to study the dynamic radio sky. Efficient pipelines are vital in such research, where searches often form a `needle in a haystack' type of problem to solve. However, the existing pipelines developed among the radio-transient community are not suitable for the scale …
Decades long monitoring of millisecond pulsars, which exhibit highly stable rotational periods, in pulsar timing array experiments is on the threshold of discovering nanohertz stochastic gravitational wave background. This paper describes the Indian Pulsar timing array (InPTA) experiment, which employs the upgraded Giant Metrewave Radio Telescope (uGMRT) for timing an ensemble of millisecond pulsars for this purpose. We highlight InPTA's observation strategies and analysis methods, which are relevant for a future PTA experiment with the more sensitive Square Kilometer Array (SKA) telescope. We…
The "Square Kilometre Array" (SKA) is a large international radio telescope project characterised, as suggested by its name, by a total collecting area of approximately one square kilometre, and consisting of several interferometric arrays to observe at metric and centimetric wavelengths. The deployment of the SKA will take place in two sites, in South Africa and Australia, and in two successive phases. From its Phase 1, the SKA will be one of the most formidable scientific machines ever deployed by mankind, and by far the most impressive in terms of data throughput and required computing powe…
As of 2023, the Square Kilometre Array will constitute the world's largest radio telescope, offering unprecedented capabilities for a diverse science programme in radio astronomy. At the same time, the SKA will be ideally suited to detect extensive air showers initiated by cosmic rays in the Earth's atmosphere via their radio emission. With its very dense and uniform antenna spacing in a fiducial area of one km$^2$ and its large bandwidth of 50-350 MHz, the low-frequency part of the SKA will provide very precise measurements of individual cosmic ray air showers. These precision measurements wi…
We present the first southern-hemisphere all-sky imager and radio-transient monitoring system implemented on two prototype stations of the low-frequency component of the Square Kilometre Array. Since its deployment the system has been used for real-time monitoring of the recorded commissioning data. Additionally, a transient searching algorithm has been executed on the resulting all-sky images. It uses a difference imaging technique, and has enabled identification of a wide variety of transient classes, ranging from human-made radio-frequency interference to genuine astrophysical events. Obser…
We present an update on the NZ-wide advances in the field of Radio Astronomy and Radio Engineering with a particular focus on contributions, not thus reported elsewhere, which hope to either directly or indirectly contribute to New Zealand's engagement with the international Square Kilometre Array (SKA) project. We discuss the status of the SKA project in New Zealand with particular reference to activities of the New Zealand Square Kilometre Array Research and Development Consortium.
The Square Kilometre Array (SKA) will make ground breaking discoveries in pulsar science. In this chapter we outline the SKA surveys for new pulsars, as well as how we will perform the necessary follow-up timing observations. The SKA's wide field-of-view, high sensitivity, multi-beaming and sub-arraying capabilities, coupled with advanced pulsar search backends, will result in the discovery of a large population of pulsars. These will enable the SKA's pulsar science goals (tests of General Relativity with pulsar binary systems, investigating black hole theorems with pulsar-black hole binaries,…
ASKAP Monitoring of the Rubin Deep Drilling Fields
University of Sydney（悉尼大学）；ARC Centre of Excellence for Gravitational Wave Discovery (OzGrav)（引力波发现卓越研究中心）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；The University of Melbourne（墨尔本大学）；Macquarie University（麦考瑞大学）
It was decided in May 2012 that the Square Kilometre Array (SKA) will be built in Africa and Australia, two Southern Hemisphere continents. Here we discuss the plan for SKA design and construction, and how New Zealand radio astronomers can participate in this project and contribute to astronomy and astrophysics research. Geodesy and the study of tectonic plate motion is another important area of research for New Zealand radio astronomy to contribute to. As New Zealand is located at the boundary between two colliding tectonic plates (Australian and Pacific) and most of geological activity in Ne…
The Square Kilometer Array (SKA) will be the largest Global science project of the next two decades. It will encompass a sensor network dedicated to radioastronomy, covering two continents. It will be constructed in remote areas of South Africa and Australia, spreading over 3000Km, in high solar irradiance latitudes. Solar Power supply is therefore an option to power supply the SKA and contribute to a zero carbon footprint next generation telescope. Here we outline the major characteristics of the SKA and some innovation approaches on thermal solar energy Integration with SKA prototypes.
One of the five key science projects for the Square Kilometre Array (SKA) is "The Origin and Evolution of Cosmic Magnetism", in which radio polarimetry will be used to reveal what cosmic magnets look like and what role they have played in the evolving Universe. Many of the SKA prototypes now being built are also targeting magnetic fields and polarimetry as key science areas. Here I review the prospects for innovative new polarimetry and Faraday rotation experiments with forthcoming facilities such as ASKAP, LOFAR, the ATA, the EVLA, and ultimately the SKA. Sensitive wide-field polarisation sur…
Deep Investigation of Neutral Gas Origins (DINGO): Options for the Processing and Storage of Radio Astronomy Data for robust Deep Spectral Line Imaging in the SKA-Era using uv-Grids
International Centre for Radio Astronomy Research, University of Western Australia（西澳大利亚大学国际无线电天文研究中心）；Australian SKA Regional Centre (AusSRC), University of Western Australia（西澳大利亚大学澳大利亚SKA区域中心）；Pawsey Supercomputing Research Centre（珀西超级计算研究中心）；Oak Ridge National Laboratory（橡树岭国家实验室）；Australia Telescope National Facility, CSIRO Space & Astronomy（CSIRO空间与天文澳大利亚望远镜国家设施）；ARC Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D)（ARC三维全天空天体物理学卓越中心）
The vast collecting area of the Square Kilometre Array (SKA), harnessed by sensitive receivers, flexible digital electronics and increased computational capacity, could permit the most sensitive and exhaustive search for technologically-produced radio emission from advanced extraterrestrial intelligence (SETI) ever performed. For example, SKA1-MID will be capable of detecting a source roughly analogous to terrestrial high-power radars (e.g. air route surveillance or ballistic missile warning radars, EIRP (EIRP = equivalent isotropic radiated power, ~10^17 erg sec^-1) at 10 pc in less than 15 m…
Densely-packed, all-digital aperture arrays form a key area of technology development required for the Square Kilometre Array (SKA) radio telescope. The design of real-time signal processing systems for digital aperture arrays is currently a central challenge in pathfinder projects worldwide. We describe interim results of such work; an heirarchical, frequency-domain beamforming architecture for synthesising a sky beam from the wideband antenna feeds of digital aperture arrays.
We present in this reference paper an instrumental project dedicated to the monitoring of solar activity during solar cycle 25. It concerns the survey of fast evolving chromospheric events implied in Space Weather, such as flares, coronal mass ejections, filament instabilities and Moreton waves. Coronal waves are produced by large flares around the solar maximum and propagate with chromospheric counterparts; they are rare, faint, difficult to observe, and for that reason, challenging. They require systematic observations with automatic, fast and multi-channel optical instruments. MeteoSpace is…
Open Science and Artificial Intelligence for supporting the sustainability of the SRC Network: The espSRC case
Instituto de Astrofísica de Andalucía (IAA), CSIC（西班牙国家研究委员会安达卢西亚天体物理研究所）；Pue Control（普埃控制公司）；Universidad de Granada（格拉纳达大学）；Safran Electronics and Defense（赛峰电子与防务公司）
Measuring High-Energy Cosmic Particles with the SKA
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；University of Manchester（曼彻斯特大学）；Vrije Universiteit Brussel (VUB)（布鲁塞尔自由大学）；Radboud University Nijmegen（奈梅亨拉德堡德大学）；Curtin University（科廷大学）；University of Groningen（格罗宁根大学）；Netherlands Institute for Radio Astronomy (ASTRON)（荷兰射电天文研究所）；Purple Mountain Observatory, Chinese Academy of Sciences（中国科学院紫金山天文台）；Nikhef（尼霍夫核物理与粒子物理研究所）；Khalifa University（哈利法大学）；Can Tho University（芹苴大学）；Nanjing University（南京大学）；Xidian University（西安电子科技大学）；University of Science and Technology of China（中国科学技术大学）
The Square Kilometre Array (SKA) is the most ambitious radio telescope ever planned. With a collecting area of about a square kilometre, the SKA will be far superior in sensitivity and observing speed to all current radio facilities. The scientific capability promised by the SKA and its technological challenges provide an ideal base for interdisciplinary research, technology transfer, and collaboration between universities, research centres and industry. The SKA in the radio regime and the European Extreme Large Telescope (E-ELT) in the optical band are on the roadmap of the European Strategy …
Blazars are extremely variable objects emitting radiation across the electromagnetic spectrum and showing variability on time scales from minutes to years. For the understanding of the emission mechanisms, simultaneous multi-wavelength observations are crucial. Various models for flares predict simultaneous flux increases in the X-ray and gamma-ray band or more complex variability patterns, depending on the dominant process responsible for the gamma-ray emission. Monitoring at TeV energies is providing important information to distinguish between different models. To study duty cycle and varia…
