Photometric redshifts for the Kilo-Degree Survey. Machine-learning analysis with artificial neural networks
M. Bilicki, H. Hoekstra, M. J. I. Brown, V. Amaro, C. Blake, S. Cavuoti, J. T. A. de Jong, C. Georgiou, H. Hildebrandt, C. Wolf, A. Amon, M. Brescia, S. Brough,… 展开作者
M. Bilicki, H. Hoekstra, M. J. I. Brown, V. Amaro, C. Blake, S. Cavuoti, J. T. A. de Jong, C. Georgiou, H. Hildebrandt, C. Wolf, A. Amon, M. Brescia, S. Brough, M. V. Costa-Duarte, T. Erben, K. Glazebrook, A. Grado, C. Heymans, T. Jarrett, S. Joudaki, K. Kuijken, G. Longo, N. Napolitano, D. Parkinson, C. Vellucci, G. A. Verdoes Kleijn, L. Wang
We present a machine-learning photometric redshift analysis of the Kilo-Degree Survey Data Release 3, using two neural-network based techniques: ANNz2 and MLPQNA. Despite limited coverage of spectroscopic training sets, these ML codes provide photo-zs of quality comparable to, if not better than, those from the BPZ code, at least up to zphot<0.9 and r<23.5. At the bright end of r<20, where very complete spectroscopic data overlapping with KiDS are available, the performance of the ML photo-zs clearly surpasses that of BPZ, currently the primary photo-z method for KiDS.
Using the Galaxy And Mass Assembly (GAMA) spectroscopic survey as calibration, we furthermore study how photo-zs improve for bright sources when photometric parameters additional to magnitudes are included in the photo-z derivation, as well as when VIKING and WISE infrared bands are added. While the fiducial four-band ugri setup gives a photo-z bias $δz=-2e-4$ and scatter $σ_z<0.022$ at mean z = 0.23, combining magnitudes, colours, and galaxy sizes reduces the scatter by ~7% and the bias by an order of magnitude. Once the ugri and IR magnitudes are joined into 12-band photometry spanning up to 12 $μ$, the scatter decreases by more than 10% over the fiducial case. Finally, using the 12 bands together with optical colours and linear sizes gives $δz<4e-5$ and $σ_z<0.019$.
This paper also serves as a reference for two public photo-z catalogues accompanying KiDS DR3, both obtained using the ANNz2 code. The first one, of general purpose, includes all the 39 million KiDS sources with four-band ugri measurements in DR3. The second dataset, optimized for low-redshift studies such as galaxy-galaxy lensing, is limited to r<20, and provides photo-zs of much better quality than in the full-depth case thanks to incorporating optical magnitudes, colours, and sizes in the GAMA-calibrated photo-z derivation.
We present a bright galaxy sample with accurate and precise photometric redshifts (photo-zs), selected using $ugriZYJHK_\mathrm{s}$ photometry from the Kilo-Degree Survey (KiDS) Data Release 4 (DR4). The highly pure and complete dataset is flux-limited at $r<20$ mag, covers $\sim1000$ deg$^2$, and contains about 1 million galaxies after artifact masking. We exploit the overlap with Galaxy And Mass Assembly (GAMA) spectroscopy as calibration to determine photo-zs with the supervised machine learning neural network algorithm implemented in the ANNz2 software. The photo-zs have mean error of $|\l…
The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products,…
We present a catalog of quasars and corresponding redshifts in the Kilo-Degree Survey (KiDS) Data Release 4. We trained machine learning (ML) models, using optical ugri and near-infrared ZYJHK_s bands, on objects known from Sloan Digital Sky Survey (SDSS) spectroscopy. We define inference subsets from the 45 million objects of the KiDS photometric data limited to 9-band detections. We show that projections of the high-dimensional feature space can be successfully used to investigate the estimations. The model creation employs two test subsets: randomly selected and the faintest objects, which …
The Kilo-Degree Survey (KiDS) is a multi-band imaging survey designed for cosmological studies from weak lensing and photometric redshifts. It uses the ESO VLT Survey Telescope with its wide-field camera OmegaCAM. KiDS images are taken in four filters similar to the SDSS ugri bands. The best-seeing time is reserved for deep r-band observations that reach a median 5-sigma limiting AB magnitude of 24.9 with a median seeing that is better than 0.7arcsec. Initial KiDS observations have concentrated on the GAMA regions near the celestial equator, where extensive, highly complete redshift catalogues…
The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the survey is mapping the large-scale matter distribution in the Universe back to a redshift of ~0.5. Secondary science cases are manifold, covering topics such as galaxy evolution, Milky Way structure, and…
By the time that the first phase of the Square Kilometre Array is deployed it will be able to perform state of the art Large Scale Structure (LSS) as well as Weak Gravitational Lensing (WGL) measurements of the distribution of matter in the Universe. In this chapter we concentrate on the synergies that result from cross-correlating these different SKA data products as well as external correlation with the weak lensing measurements available from CMB missions. We show that the Dark Energy figures of merit obtained individually from WGL/LSS measurements and their independent combination is signi…
We present a method that accurately propagates residual uncertainties in photometric redshift distributions into the cosmological inference from weak lensing measurements. The redshift distributions of tomographic redshift bins are parameterised using a flexible modified Gaussian mixture model. We fit this model to pre-calibrated redshift distributions and implement an analytic marginalisation over the potentially several hundred redshift nuisance parameters in the weak lensing likelihood, which is demonstrated to accurately recover the cosmological posterior. By iteratively fitting cosmologic…
Exploiting the full statistical power of future cosmic shear surveys will necessitate improvements to the accuracy with which the gravitational lensing signal is measured. We present a framework for calibrating shear with image simulations that demonstrates the importance of including realistic correlations between galaxy morphology, size and more importantly, photometric redshifts. This realism is essential so that selection and shape measurement biases can be calibrated accurately for a tomographic cosmic shear analysis. We emulate Kilo-Degree Survey (KiDS) observations of the COSMOS field u…
We forecast future constraints on local-type primordial non-Gaussianity parameter $f_{\rm NL}$ with a photometric galaxy survey by Euclid, a continuum galaxy survey by Square Kilometre Array (SKA), and their combination. We derive a general expression for the covariance matrix of the power spectrum estimates of multiple tracers to show how the so-called multitracer technique improves constraints on $f_{\rm NL}$. In particular we clarify the role of the overlap fraction of multiple tracers and the division method of the tracers. Our Fisher matrix analysis indicates that stringent constraints of…
National Centre for Nuclear Research（国家核研究中心）；Leiden University（莱顿大学）；Center for Theoretical Physics, Polish Academy of Sciences（波兰科学院理论物理中心）；Jagiellonian University（雅盖隆大学）；Ludwig-Maximilians-Universität（路德维希-马克西米利安大学）；INAF – Osservatorio Astronomico di Capodimonte（意大利国家天体物理研究所–卡波迪蒙特天文台）；Sun Yat-sen University（中山大学）
We present a catalog of quasars selected from broad-band photometric ugri data of the Kilo-Degree Survey Data Release 3 (KiDS DR3). The QSOs are identified by the random forest (RF) supervised machine learning model, trained on SDSS DR14 spectroscopic data. We first cleaned the input KiDS data from entries with excessively noisy, missing or otherwise problematic measurements. Applying a feature importance analysis, we then tune the algorithm and identify in the KiDS multiband catalog the 17 most useful features for the classification, namely magnitudes, colors, magnitude ratios, and the stella…
We use the overlap between multiband photometry of the Kilo-Degree Survey (KiDS) and spectroscopic data based on the Sloan Digital Sky Survey (SDSS) and Galaxy And Mass Assembly (GAMA) to infer the colour-magnitude relation of red-sequence galaxies. We then use this inferred relation to select luminous red galaxies (LRGs) in the redshift range of $0.1<z<0.7$ over the entire KiDS Data Release 3 footprint. We construct two samples of galaxies with different constant comoving densities and different luminosity thresholds. The selected red galaxies have photometric redshifts with typical photo-z e…
The European Southern Observatory's (ESO) Visible and Infrared Survey Telescope for Astronomy (VISTA) is a 4-m class survey telescope for wide-field near-infrared imaging. VISTA is currently running a suite of six public surveys, which will shortly deliver their first Europe wide public data releases to ESO. The VISTA Kilo-degree Infrared Galaxy Survey (VIKING) forms a natural intermediate between current wide shallow, and deeper more concentrated surveys, by targeting two patches totalling 1500 sq.deg in the northern and southern hemispheres with measured 5-sigma limiting depths of Z ~ 22.4, …
The masses of galaxy clusters can be measured using data obtained exclusively from wide photometric surveys in one of two ways: directly from the amplitude of the weak lensing signal or, indirectly, through the use of scaling relations calibrated using binned lensing measurements. In this paper, we build on a recently proposed idea and implement an alternative method based on the radial profile of the satellite distribution. This technique relies on splashback, a feature associated with the apocenter of recently accreted galaxies that offers a clear window into the phase-space structure of clu…
We present The-wiZZ, an open source and user-friendly software for estimating the redshift distributions of photometric galaxies with unknown redshifts by spatially cross-correlating them against a reference sample with known redshifts. The main benefit of The-wiZZ is in separating the angular pair finding and correlation estimation from the computation of the output clustering redshifts allowing anyone to create a clustering redshift for their sample without the intervention of an "expert". It allows the end user of a given survey to select any sub-sample of photometric galaxies with unknown …
Next-generation surveys like the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory will generate orders of magnitude more discoveries of transients and variable stars than previous surveys. To prepare for this data deluge, we developed the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC), a competition which aimed to catalyze the development of robust classifiers under LSST-like conditions of a non-representative training set for a large photometric test set of imbalanced classes. Over 1,000 teams participated in PLAsTiCC, which was hosted in …
We present a determination of the effects of including galaxy morphological parameters in photometric redshift estimation with an artificial neural network method. Neural networks, which recognize patterns in the information content of data in an unbiased way, can be a useful estimator of the additional information contained in extra parameters, such as those describing morphology, if the input data are treated on an equal footing. We use imaging and five band photometric magnitudes from the All-wavelength Extended Groth Strip International Survey. It is shown that certain principal components…
The scientific value of the next generation of large continuum surveys would be greatly increased if the redshifts of the newly detected sources could be rapidly and reliably estimated. Given the observational expense of obtaining spectroscopic redshifts for the large number of new detections expected, there has been substantial recent work on using machine learning techniques to obtain photometric redshifts. Here we compare the accuracy of the predicted photometric redshifts obtained from Deep Learning(DL) with the k-Nearest Neighbour (kNN) and the Decision Tree Regression (DTR) algorithms. W…
We present the curation and verification of a new combined optical and near infrared dataset for cosmology and astrophysics, derived from the combination of $ugri$-band imaging from the Kilo Degree Survey (KiDS) and $ZY\!J\!H\!K_{\rm s}$-band imaging from the VISTA Kilo degree Infrared Galaxy (VIKING) survey. This dataset is unrivaled in cosmological imaging surveys due to its combination of area ($458$ deg$^2$ before masking), depth ($r\le25$), and wavelength coverage ($ugriZY\!J\!H\!K_{\rm s}$). The combination of survey depth, area, and (most importantly) wavelength coverage allows signific…
In this paper, we present the tools used to search for galaxy clusters in the Kilo Degree Survey (KiDS), and our first results. The cluster detection is based on an implementation of the optimal filtering technique that enables us to identify clusters as over-densities in the distribution of galaxies using their positions on the sky, magnitudes, and photometric redshifts. The contamination and completeness of the cluster catalog are derived using mock catalogs based on the data themselves. The optimal signal to noise threshold for the cluster detection is obtained by randomizing the galaxy pos…
Forthcoming astronomical surveys are expected to detect new sources in such large numbers that measuring their spectroscopic redshift measurements will be not be practical. Thus, there is much interest in using machine learning to yield the redshift from the photometry of each object. We are particularly interested in radio sources (quasars) detected with the Square Kilometre Array and have found Deep Learning, trained upon a large optically-selected sample of quasi-stellar objects, to be effective in the prediction of the redshifts in three external samples of radio-selected sources. However,…
The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope, specifically designed for measuring weak gravitational lensing by galaxies and large-scale structure. When completed it will consist of 1350 square degrees imaged in four filters (ugri). Here we present the fourth public data release which more than doubles the area of sky covered by data release 3. We also include aperture-matched ZYJHKs photometry from our partner VIKING survey on the VISTA telescope in the photometry catalogue. We illustrate the data quality an…
We study projected underdensities in the cosmic galaxy density field known as 'troughs', and their overdense counterparts, which we call 'ridges'. We identify these regions using a bright sample of foreground galaxies from the photometric Kilo-Degree Survey (KiDS), specifically selected to mimic the spectroscopic Galaxy And Mass Assembly survey (GAMA). Using background galaxies from KiDS, we measure the weak gravitational lensing profiles of the troughs/ridges. We quantify the amplitude of their lensing strength $A$ as a function of galaxy density percentile rank $P$ and galaxy overdensity $δ$…
The bubble size distribution of ionized hydrogen regions probes the information about the morphology of \HII\ bubbles during the reionization. Conventionally, the \HII\ bubble size distribution can be derived from the tomographic imaging data of the redshifted 21~cm signal from the epoch of reionization, which, however, is observationally challenging even for the upcoming large radio interferometer arrays. Given that these interferometers promise to measure the 21~cm power spectrum accurately, we propose a new method, which is based on the artificial neural networks (ANN), to reconstruct the \…
We present a comparison between Gaussian processes (GPs) and artificial neural networks (ANNs) as methods for determining photometric redshifts for galaxies, given training set data. In particular, we compare their degradation in performance as the training set size is degraded in ways which might be caused by the observational limitations of spectroscopy. We find that performance with large, complete training sets is very similar, although the ANN achieves slightly smaller root mean square errors. If the size of the training set is reduced by random sampling, the RMS errors of both methods in…
We present a new technique to estimate the level of contamination between photometric redshift bins. If the true angular cross-correlation between redshift bins can be safely assumed to be zero, any measured cross-correlation is a result of contamination between the bins. We present the theory for an arbitrary number of redshift bins, and discuss in detail the case of two and three bins which can be easily solved analytically. We use mock catalogues constructed from the Millennium Simulation to test the method, showing that artificial contamination can be successfully recovered with our method…
Automated photometric supernova classification has become an active area of research in recent years in light of current and upcoming imaging surveys such as the Dark Energy Survey (DES) and the Large Synoptic Survey Telescope, given that spectroscopic confirmation of type for all supernovae discovered will be impossible. Here, we develop a multi-faceted classification pipeline, combining existing and new approaches. Our pipeline consists of two stages: extracting descriptive features from the light curves and classification using a machine learning algorithm. Our feature extraction methods va…
We present ANNz2, a new implementation of the public software for photometric redshift (photo-z) estimation of Collister and Lahav (2004), which now includes generation of full probability distribution functions (PDFs). ANNz2 utilizes multiple machine learning methods, such as artificial neural networks and boosted decision/regression trees. The objective of the algorithm is to optimize the performance of the photo-z estimation, to properly derive the associated uncertainties, and to produce both single-value solutions and PDFs. In addition, estimators are made available, which mitigate possib…
We present a catalogue of galaxy photometric redshifts and k-corrections for the Sloan Digital Sky Survey Seven Data Release (SDSS-DR7), available on the World Wide Web. The photometric redshifts were estimated with an artificial neural network using five ugriz bands, concentration indices and Petrosian radii in the g and r bands. We have explored our redshift estimates with different training set concluding that the best choice to improve redshift accuracy comprises the Main Galaxies Sample (MGS), the Luminous Red Galaxies, and galaxies of active galactic nuclei covering the redshift range 0<…
Large sets of objects with spectroscopic redshift measurements will be needed for imaging dark energy experiments to achieve their full potential, serving two goals:_training_, i.e., the use of objects with known redshift to develop and optimize photometric redshift algorithms; and_calibration_, i.e., the characterization of moments of redshift (or photo-z error) distributions. Better training makes cosmological constraints from a given experiment stronger, while highly-accurate calibration is needed for photo-z systematics not to dominate errors. In this white paper, we investigate the requir…
Baryonic Acoustic Oscillations (BAO) and their effects on the matter power spectrum can be studied by using the Lyman-alpha absorption signature of the matter density field along quasar (QSO) lines of sight. A measurement sufficiently accurate to provide useful cosmological constraints requires the observation of ~100000 quasars in the redshift range 2.2<z<3.5 over ~8000 deg2. Such a survey is planned by the Baryon Oscillation Spectroscopic Survey (BOSS) project of the Sloan Digital Sky Survey (SDSS-III).In practice, one needs a stellar rejection of more than two orders of magnitude with a sel…
We present a tomographic cosmic shear analysis of the Kilo-Degree Survey (KiDS) combined with the VISTA Kilo-Degree Infrared Galaxy Survey (VIKING). This is the first time that a full optical to near-infrared data set has been used for a wide-field cosmological weak lensing experiment. This unprecedented data, spanning $450~$deg$^2$, allows us to improve significantly the estimation of photometric redshifts, such that we are able to include robustly higher-redshift sources for the lensing measurement, and - most importantly - solidify our knowledge of the redshift distributions of the sources.…
