Anomaly detection in the Zwicky Transient Facility DR3
K. L. Malanchev, M. V. Pruzhinskaya, V. S. Korolev, P. D. Aleo, M. V. Kornilov, E. E. O. Ishida, V. V. Krushinsky, F. Mondon, S. Sreejith, A. A. Volnova, A. A. … 展开作者
K. L. Malanchev, M. V. Pruzhinskaya, V. S. Korolev, P. D. Aleo, M. V. Kornilov, E. E. O. Ishida, V. V. Krushinsky, F. Mondon, S. Sreejith, A. A. Volnova, A. A. Belinski, A. V. Dodin, A. M. Tatarnikov, S. G. Zheltoukhov
备注：
31 pages, 28 figures, 4 tables, accepted for publication in MNRAS
We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of 3 stages: feature extraction, search of outliers with machine learning algorithms and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million objects. A set of 4 automatic learning algorithms was used to identify 277 outliers, which were subsequently scrutinised by an expert. From these, 188 (68%) were found to be bogus light curves -- including effects from the image subtraction pipeline as well as overlapping between a star and a known asteroid, 66 (24%) were previously reported sources whereas 23 (8%) correspond to non-catalogued objects, with the two latter cases of potential scientific interest (e. g. 1 spectroscopically confirmed RS Canum Venaticorum star, 4 supernovae candidates, 1 red dwarf flare). Moreover, using results from the expert analysis, we were able to identify a simple bi-dimensional relation which can be used to aid filtering potentially bogus light curves in future studies. We provide a complete list of objects with potential scientific application so they can be further scrutinised by the community. These results confirm the importance of combining automatic machine learning algorithms with domain knowledge in the construction of recommendation systems for astronomy. Our code is publicly available at https://github.com/snad-space/zwad
The Zwicky Transient Facility (ZTF) is currently surveying the entire northern sky, including dense Galactic plane fields. Here, we present preliminary results of the search for gravitational microlensing events in the ZTF data collected from the beginning of the survey (March 20, 2018) through June 30, 2019.
The Zwicky Transient Facility (ZTF) is a next-generation optical synoptic survey that builds on the experience and infrastructure of the Palomar Transient Factory (PTF). Using a new 47 deg$^2$ survey camera, ZTF will survey more than an order of magnitude faster than PTF to discover rare transients and variables. I describe the survey and the camera design. Searches for young supernovae, fast transients, counterparts to gravitational-wave detections, and rare variables will benefit from ZTF's high cadence, wide area survey.
The Zwicky Transient Facility (ZTF), a state-of-the-art optical robotic sky survey, registers on the order of a million transient events - such as supernova explosions, changes in brightness of variable sources, or moving object detections - every clear night, and generates associated real-time alerts. We present Alert-Classifying Artificial Intelligence (ACAI), an open-source deep-learning framework for the phenomenological classification of ZTF alerts. ACAI uses a set of five binary classifiers to characterize objects which, in combination with the auxiliary/contextual event information avai…
The Zwicky Transient Facility (ZTF) is conducting a nightly public survey of all 13 TESS northern sectors in 2019-2020. ZTF will observe the portions of the current TESS sectors visible from Palomar Observatory each night. Each ZTF pointing will have one exposure each with $g$ and $r$ filters, totaling two images per night. ZTF is also making additional nightly $g$- and $r$-band observations of denser stellar regions (e.g. near the Galactic Plane) to better facilitate variability studies of Galactic objects. The limiting magnitude of the ZTF observations is $\approx$20.6 and ZTF saturates at m…
We introduce a new method for analyzing sparse photometric data of asteroids and apply it to Zwicky Transient Facility observations of the Jupiter Trojan asteroids. The method relies on the creation of a likelihood model that includes the probability distribution of rotational brightness variations at an unknown rotation phase. The likelihood model is analyzed via a Markov Chain Monte Carlo to quantify the uncertainty in our parameter estimates. Using this method, we provide color, phase parameter, absolute magnitude and amplitude of rotation measurements for 1049 Jupiter Trojans. We find that…
Efficient automated detection of flux-transient, reoccurring flux-variable, and moving objects is increasingly important for large-scale astronomical surveys. We present braai, a convolutional-neural-network, deep-learning real/bogus classifier designed to separate genuine astrophysical events and objects from false positive, or bogus, detections in the data of the Zwicky Transient Facility (ZTF), a new robotic time-domain survey currently in operation at the Palomar Observatory in California, USA. Braai demonstrates a state-of-the-art performance as quantified by its low false negative and fa…
We present a novel algorithm for scheduling the observations of time-domain imaging surveys. Our Integer Linear Programming approach optimizes an observing plan for an entire night by assigning targets to temporal blocks, enabling strict control of the number of exposures obtained per field and minimizing filter changes. A subsequent optimization step minimizes slew times between each observation. Our optimization metric self-consistently weights contributions from time-varying airmass, seeing, and sky brightness to maximize the transient discovery rate. We describe the implementation of this …
$\texttt{BTSbot}$: A Multi-input Convolutional Neural Network to Automate and Expedite Bright Transient Identification for the Zwicky Transient Facility
Electromagnetic follow-up of gravitational wave detections is very resource intensive, taking up hours of limited observation time on dozens of telescopes. Creating more efficient schedules for follow-up will lead to a commensurate increase in counterpart location efficiency without using more telescope time. Widely used in operations research and telescope scheduling, mixed integer linear programming (MILP) is a strong candidate to produce these higher-efficiency schedules, as it can make use of powerful commercial solvers that find globally optimal solutions to provided problems . We detail …
The Zwicky Transient Facility is a new robotic-observing program, in which a newly engineered 600-MP digital camera with a pioneeringly large field of view, 47~square degrees, will be installed into the 48-inch Samuel Oschin Telescope at the Palomar Observatory. The camera will generate $\sim 1$~petabyte of raw image data over three years of operations. In parallel related work, new hardware and software systems are being developed to process these data in real time and build a long-term archive for the processed products. The first public release of archived products is planned for early 2019…
We present DeepStreaks, a convolutional-neural-network, deep-learning system designed to efficiently identify streaking fast-moving near-Earth objects that are detected in the data of the Zwicky Transient Facility (ZTF), a wide-field, time-domain survey using a dedicated 47 sq. deg camera attached to the Samuel Oschin 48-inch Telescope at the Palomar Observatory in California, United States. The system demonstrates a 96-98% true positive rate, depending on the night, while keeping the false positive rate below 1%. The sensitivity of DeepStreaks is quantified by the performance on the test data…
Microlensing surveys have discovered thousands of events with almost all events discovered within the Galactic bulge or toward the Magellanic clouds. The Zwicky Transient Facility (ZTF), while not designed to be a microlensing campaign, is an optical time-domain survey that observes the entire northern sky every few nights including the Galactic plane. ZTF observes $\sim10^9$ stars in g-band and r-band and can significantly contribute to the observed microlensing population. We predict that ZTF will observe $\sim$1100 microlensing events in three years of observing within $10^\circ$ degrees la…
The Zwicky Transient Facility is a time-domain optical survey that has substantially increased our ability to observe and construct massive catalogs of astronomical objects by use of its 47 square degree camera that can observe in multiple filters. However the telescope's i-band filter suffers from significant atmospheric fringes that reduce photometric precision, especially for faint sources and in multi-epoch co-additions. Here we present a method for constructing models of these atmospheric fringes using Principal Component Analysis that can be used to identify and remove these artifacts fr…
The Zwicky Transient Facility (ZTF) Observing System (OS) is the data collector for the ZTF project to study astrophysical phenomena in the time domain. ZTF OS is based upon the 48-inch aperture Schmidt-type design Samuel Oschin Telescope at the Palomar Observatory in Southern California. It incorporates new telescope aspheric corrector optics, dome and telescope drives, a large-format exposure shutter, a flat-field illumination system, a robotic bandpass filter exchanger, and the key element: a new 47-square-degree, 600 megapixel cryogenic CCD mosaic science camera, along with supporting equi…
When planning a survey for astronomical transients, many factors such as cadence, filter choice, sky coverage, and depth of observations need to be balanced in order to optimize the scientific gain of the survey. Here we present a software package called $\texttt{simsurvey}$ for simulating the supernova lightcurves that are expected based on a survey strategy, which can then be used to determine the potential for discoveries of each strategy in question. The code is set up in a modular fashion that allows easy modification of small details of the survey and enables the user to adapt it to any …
There is a growing concern about an impact of low-Earth-orbit (LEO) satellite constellations on ground-based astronomical observations, in particular, on wide-field surveys in the optical and infrared. The Zwicky Transient Facility (ZTF), thanks to the large field of view of its camera, provides an ideal setup to study the effects of LEO megaconstellations - such as SpaceX's Starlink - on astronomical surveys. Here, we analyze the archival ZTF observations collected between 2019 November and 2021 September and find 5301 satellite streaks that can be attributed to Starlink satellites. We find t…
We describe ZStreak, a semi-real-time pipeline specialized in detecting small, fast-moving near-Earth asteroids (NEAs) that is currently operating on the data from the newly-commissioned Zwicky Transient Facility (ZTF) survey. Based on a prototype originally developed by Waszczak et al. (2017) for the Palomar Transient Factory (PTF), the predecessor of ZTF, ZStreak features an improved machine-learning model that can cope with the $10\times$ data rate increment between PTF and ZTF. Since its first discovery on 2018 February 5 (2018 CL), ZTF/ZStreak has discovered $45$ confirmed new NEAs over a…
The Zwicky Transient Facility (ZTF) survey generates real-time alerts for optical transients, variables, and moving objects discovered in its wide-field survey. We describe the ZTF alert stream distribution and processing (filtering) system. The system uses existing open-source technologies developed in industry: Kafka, a real-time streaming platform, and Avro, a binary serialization format. The technologies used in this system provide a number of advantages for the ZTF use case, including (1) built-in replication, scalability, and stream rewind for the distribution mechanism; (2) structured m…
California Institute of Technology（加州理工学院）；IPAC, California Institute of Technology（加州理工学院IPAC）；University of Maryland（马里兰大学）；Center for Data Driven Discovery, California Institute of Technology（加州理工学院数据驱动发现中心）；University of Washington（华盛顿大学）；University of Minnesota（明尼苏达大学）；Caltech Optical Observatories, California Institute of Technology（加州理工学院光学天文台）
We present Tails, an open-source deep-learning framework for the identification and localization of comets in the image data of the Zwicky Transient Facility (ZTF), a robotic optical time-domain survey currently in operation at the Palomar Observatory in California, USA. Tails employs a custom EfficientDet-based architecture and is capable of finding comets in single images in near real time, rather than requiring multiple epochs as with traditional methods. The system achieves state-of-the-art performance with 99% recall, 0.01% false positive rate, and 1-2 pixel root mean square error in the …
The discovery rate of fast radio bursts (FRBs) is increasing dramatically thanks to new radio facilities. Meanwhile, wide-field instruments such as the 47 deg$^2$ Zwicky Transient Facility (ZTF) survey the optical sky to study transient and variable sources. We present serendipitous ZTF observations of the CHIME repeating source FRB 180916.J0158+65, that was localized to a spiral galaxy 149 Mpc away and is the first FRB suggesting periodic modulation in its activity. While 147 ZTF exposures corresponded to expected high-activity periods of this FRB, no single ZTF exposure was at the same time …
The Zwicky Transient Facility is a large optical survey in multiple filters producing hundreds of thousands of transient alerts per night. We describe here various machine learning (ML) implementations and plans to make the maximal use of the large data set by taking advantage of the temporal nature of the data, and further combining it with other data sets. We start with the initial steps of separating bogus candidates from real ones, separating stars and galaxies, and go on to the classification of real objects into various classes. Besides the usual methods (e.g., based on features extracte…
Detections of coalescing binary black holes by LIGO have opened a new window of transient astronomy. With increasing sensitivity of LIGO and participation of the Virgo detector in Cascina, Italy, we expect to soon detect coalescence of compact binary systems with one or more neutron stars. These are the prime targets for electromagnetic follow-up of gravitational wave triggers, which holds enormous promise of rich science. However, hunting for electromagnetic counterparts of gravitational wave events is a non-trivial task due to the sheer size of the error regions, which could span hundreds of…
The Zwicky Transient Facility (ZTF) is a new robotic time-domain survey currently in progress using the Palomar 48-inch Schmidt Telescope. ZTF uses a 47 square degree field with a 600 megapixel camera to scan the entire northern visible sky at rates of ~3760 square degrees/hour to median depths of g ~ 20.8 and r ~ 20.6 mag (AB, 5sigma in 30 sec). We describe the Science Data System that is housed at IPAC, Caltech. This comprises the data-processing pipelines, alert production system, data archive, and user interfaces for accessing and analyzing the products. The realtime pipeline employs a nov…
The Zwicky Transient Facility Bright Transient Survey. III. $\texttt{BTSbot}$: Automated Identification and Follow-up of Bright Transients with Deep Learning
The Zwicky Transient Facility (ZTF) is a new optical time-domain survey that uses the Palomar 48-inch Schmidt telescope. A custom-built wide-field camera provides a 47 deg$^2$ field of view and 8 second readout time, yielding more than an order of magnitude improvement in survey speed relative to its predecessor survey, the Palomar Transient Factory (PTF). We describe the design and implementation of the camera and observing system. The ZTF data system at the Infrared Processing and Analysis Center provides near-real-time reduction to identify moving and varying objects. We outline the analysi…
Photometric classification of supernovae detected by the Zwicky Transient Facility using noise augmentation
Humboldt-Universität zu Berlin（柏林洪堡大学）；Deutsches Elektronen-Synchrotron（德国电子同步辐射加速器中心）；Leibniz-Institut für Astrophysik Potsdam (AIP)（波茨坦阿基米德天体物理研究所）；European Southern Observatory（欧洲南方天文台）；University of Washington（华盛顿大学）；Trinity College Dublin, The University of Dublin（都柏林三一学院）；California Institute of Technology（加州理工学院）；National Central University（中央大学）；Lancaster University（兰卡斯特大学）；Institute of Space Sciences (ICE, CSIC)（空间科学研究所）；Institut d’Estudis Espacials de Catalunya (IEEC)（加泰罗尼亚空间研究院）
We present a systematic search for optical counterparts to 13 gravitational wave (GW) triggers involving at least one neutron star during LIGO/Virgo's third observing run. We searched binary neutron star (BNS) and neutron star black hole (NSBH) merger localizations with the Zwicky Transient Facility (ZTF) and undertook follow-up with the Global Relay of Observatories Watching Transients Happen (GROWTH) collaboration. The GW triggers had a median localization of 4480 deg^2, median distance of 267 Mpc and false alarm rates ranging from 1.5 to 1e-25 per yr. The ZTF coverage had a median enclosed …
Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility
Department of Physics and Astronomy, Northwestern University（西北大学物理与天文学系）；Center for Interdisciplinary Exploration and Research in Astrophysics (CIERA)（天体物理跨学科探索与研究中心）；NSF-Simons AI Institute for the Sky (SkAI)（NSF-西蒙斯天空人工智能研究所）；School of Physics and Astronomy, University of Minnesota（明尼苏达大学物理与天文学学院）；Division of Physics, Mathematics, and Astronomy, California Institute of Technology（加州理工学院物理、数学与天文学部）
The Zwicky Transient Facility (ZTF), a public-private enterprise, is a new time domain survey employing a dedicated camera on the Palomar 48-inch Schmidt telescope with a 47 deg$^2$ field of view and 8 second readout time. It is well positioned in the development of time domain astronomy, offering operations at 10% of the scale and style of the Large Synoptic Survey Telescope (LSST) with a single 1-m class survey telescope. The public surveys will cover the observable northern sky every three nights in g and r filters and the visible Galactic plane every night in g and r. Alerts generated by t…
The periodic pulsations of stars teach us about their underlying physical process. We present a convolutional autoencoder-based pipeline as an automatic approach to search for out-of-distribution anomalous periodic variables within The Zwicky Transient Facility Catalog of Periodic Variable Stars (ZTF CPVS). We use an isolation forest to rank each periodic variable by its anomaly score. Our overall most anomalous events have a unique physical origin: they are mostly highly variable and irregular evolved stars. Multiwavelength data suggest that they are most likely Red Giant or Asymptotic Giant …
Using the second data release from the Zwicky Transient Facility (ZTF, Bellm et al. 2019), Chen et al. (2020) created a ZTF Catalog of Periodic Variable Stars (ZTF CPVS) of 781, 602 periodic variables stars (PVSs) with 11 class labels. Here, we provide a new classification model of PVSs in the ZTF CPVS using a convolutional variational autoencoder and hierarchical random forest. We cross-match the sky-coordinate of PVSs in the ZTF CPVS with those presented in the SIMBAD catalog. We identify non-stellar objects that are not previously classified, including extragalactic objects such as Quasi-St…
In this work, we propose several enhancements to a geometric transformation based model for anomaly detection in images (GeoTranform). The model assumes that the anomaly class is unknown and that only inlier samples are available for training. We introduce new filter based transformations useful for detecting anomalies in astronomical images, that highlight artifact properties to make them more easily distinguishable from real objects. In addition, we propose a transformation selection strategy that allows us to find indistinguishable pairs of transformations. This results in an improvement of…
The Zwicky Transit Factory (ZTF) is a powerful time domain survey facility with a large field of view. We apply the synthetic tracking technique to integrate a ZTF's long-dwell dataset, which consists of 133 nominal 30-second exposure frames spanning about 1.5 hours, to search for slowly moving asteroids down to approximately 23rd magnitude. We found more than one thousand objects from searching 40 CCD-quadrant subfields, each of which covers a field size of $\sim$0.73 deg$^2$. While most of the objects are main belt asteroids, there are asteroids belonging to families of Trojan, Hilda, Hungar…
In March 2017, the Intermediate Palomar Transient Factory (iPTF) ceased operations. I take this occasion to review the scientific returns from iPTF and its predecessor survey, the Palomar Transient Factory (PTF), and to summarize the lessons learned. Succeeding iPTF on the Palomar Observatory 48-inch Schmidt telescope is the Zwicky Transient Facility (ZTF), a new survey with an order of magnitude faster survey speed that is now being commissioned. I describe the design and scientific rationale for ZTF. ZTF is prototyping new alert stream technologies being explored by the Large Synoptic Survey…
Multi-Class Deep SVDD: Anomaly Detection Approach in Astronomy with Distinct Inlier Categories
Universidad de Concepción（康塞普西翁大学）；Millennium Institute of Astrophysics (MAS)（千年天体物理研究所（MAS））；Universidad of Valparaíso（瓦尔帕莱索大学）；Universidad de Chile（智利大学）；European Southern Observatory（欧洲南方天文台）
Lomonosov Moscow State University（莫斯科国立大学）；National Research University Higher School of Economics（俄罗斯高等经济大学）；Ural Federal University（乌拉尔联邦大学）；Carnegie Mellon University（卡内基梅隆大学）
A Classifier-Based Approach to Multi-Class Anomaly Detection Applied to Astronomical Time-Series
Kavli Institute for Astrophysics and Space Research, Massachusetts Institute of Technology（麻省理工学院卡弗里天体物理与空间研究所）；Irvington High School（欧文顿高中）；Department of Physics and Astronomy, University of the Western Cape（西开普大学物理与天文学系）；South African Radio Astronomy Observatory（南非射电天文观测台）
A Classifier-Based Approach to Multi-Class Anomaly Detection for Astronomical Transients
Irvington High School（欧文顿高中）；University of the Western Cape（西开普大学）；South African Radio Astronomy Observatory（南非射电天文观测台）；Massachusetts Institute of Technology（麻省理工学院）
The classic classification scheme for Active Galactic Nuclei (AGNs) was recently challenged by the discovery of the so-called changing-state (changing-look) AGNs (CSAGNs). The physical mechanism behind this phenomenon is still a matter of open debate and the samples are too small and of serendipitous nature to provide robust answers. In order to tackle this problem, we need to design methods that are able to detect AGN right in the act of changing-state. Here we present an anomaly detection (AD) technique designed to identify AGN light curves with anomalous behaviors in massive datasets. The m…
