NASA/IPAC河外星系数据库(NED)部署了一种新的基于规则的交叉匹配算法,称为匹配专家(MatchEx),能够交叉匹配具有超过1000万个对象的超大型目录(VLC)。MatchEx超越了传统的基于位置的交叉匹配算法,它使用其他可用数据和专家逻辑来确定哪个候选匹配是最好的。此外,源的局部背景密度被用来确定和最小化的假阳性匹配率和估计匹配的完整性。每个匹配决策的逻辑结果和统计概率存储在数据库中,并且可以用于调整算法和调整匹配参数阈值。对于我们的第一次生产运行,我们交叉匹配了GALEX全天巡天目录(GASC),其中包含近4000万个NUV检测到的源,与NED中的1.8亿个对象的目录。在7.5弧秒半径内为每个GASC源确定候选匹配。这些候选人被过滤基于位置的匹配概率,以及其他标准,包括对象类型和对象名称。我们估计匹配完整性为97.6%,匹配准确率为99.75%。MatchEx被用于将超过20亿个目录源与NED交叉匹配,包括Spitzer Source List、2 MASS Point-Source Catalog、AllWISE和SDSS DR 10。它还将加快作为NED文献管道的一部分的资源的常规交叉匹配。
英文摘要:
The NASA/IPAC Extragalactic Database (NED) has deployed a new rule-based cross-matching algorithm called Match Expert (MatchEx), capable of cross-matching very large catalogs (VLCs) with >10 million objects. MatchEx goes beyond traditional position-based cross-matching algorithms by using other available data together with expert logic to determine which candidate match is the best. Furthermore, the local background density of sources is used to determine and minimize the false-positive match rate and to estimate match completeness. The logical outcome and statistical probability of each match decision is stored in the database, and may be used to tune the algorithm and adjust match parameter thresholds. For our first production run, we cross-matched the GALEX All Sky Survey Catalog (GASC), containing nearly 40 million NUV-detected sources, against a directory of 180 million objects in NED. Candidate matches were identified for each GASC source within a 7.5 arcsecond radius. These candidates were filtered on position-based matching probability, and on other criteria including object type and object name. We estimate a match completeness of 97.6% and a match accuracy of 99.75%. MatchEx is being used to cross-match over 2 billion catalog sources to NED, including the Spitzer Source List, the 2MASS Point-Source Catalog, AllWISE, and SDSS DR 10. It will also speed up routine cross-matching of sources as part of the NED literature pipeline.
The Chandra-Gaia Catalog of Counterparts: Resolving ambiguous Gaia matches to X-ray sources in the Chandra Source Catalog using Machine Learning
Center for Astrophysics Harvard \& Smithsonian, 60 Garden St, Cambridge MA 02138, USA;Harvard John A. Paulson School of Engineering;Universidad del Rosario, School of Engineering, Science;The NSF AI Institute for Artificial Intelligence;New York University, Courant Institute, 60 5th Avenue, New York NY, USA;Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213;New College of Florida, 5800 Bayshore Road, Sarasota, FL 34243, USA;Astrophysics Laboratory, 3251 Hanover St, Palo Alto, CA 94304, USA
Gaia Data Release 2. Cross-match with external catalogues - Algorithms and results
上下文虽然盖亚目录本身是一个非常强大的工具,但正是这种高精度档案与其他档案的结合,才真正为天文学研究开辟了惊人的可能性。归档的高级互操作基于交叉匹配,让用户感觉就像使用单个数据归档一样。数据检索不仅应跨数据归档工作,还应跨波长域工作。无缝访问数据的第一步是计算Gaia和外部调查之间的交叉匹配。目的。我们描述了盖亚数据发布2(DR 2)目录与密集调查(泛星DR 1,2 MASS,SDSS DR 9,GSC 2.3,URAT-1,allWISE,PPMXL和APASS DR 9)和稀疏目录(Hipparcos 2,Tycho-2和RAVE 5)的预计算交叉匹配所采用的算法和结果。方法.一种新的算法是专门为稀疏目录。详细描述了DR 1所采用算法的改进和变化。结果交叉匹配的输出是官方Gaia DR 2目录的一部分。交叉匹配结果的全局分析也被提出。
Clustering Confuses Spectro-photometry: An Investigation of 2D and 3D Forced Profile Matching for Stacking Line-intensity Mapping Data on Source Catalogues
Cornell University(康奈尔大学);California Institute of Technology(加州理工学院)
AXS: A framework for fast astronomical data processing based on Apache Spark
我们介绍AXS(Astronomy eXtensions for Spark),这是一个基于Apache Spark的可扩展开源天文数据分析框架,Apache Spark是一个广泛使用的行业标准大数据处理引擎。基于Spark中的功能,AXS旨在使用熟悉的Python/AstroPy概念,DataFrame API和SQL语句查询和分析几乎任意大的天文目录。我们通过i)为Spark添加支持以实现高效的在线位置交叉匹配,ii)提供支持天文数据分析常用操作的Python库。为了支持可扩展的交叉匹配,我们开发了一种ZONES算法(Gray et al. 2004)的变体,能够在分布式、无共享架构中运行。我们将其与数据分区方案相结合,该方案可以实现快速目录交叉匹配,并处理深空数据集中经常出现的数据倾斜。交叉匹配和其他常用功能通过易于使用的Python API向最终用户公开。我们展示了AXS在SDSS、ZTF、Gaia DR 2和AllWise目录上的技术和科学性能。使用AXS,我们能够在大约30秒内对Gaia DR 2(18亿行)和AllWise(9亿行)数据集进行动态交叉匹配。我们将讨论像AXS这样的云就绪分布式系统如何提供一种自然的方式来实现对LSST等大型数据集的全面最终用户分析。
National Astronomical Observatories, Chinese Academy of Sciences(中国科学院国家天文台);National Space Science Center, Chinese Academy of Sciences(中国科学院国家空间科学中心);The Shanghai Institute of Technical Physics of the Chinese Academy of Sciences(中国科学院上海技术物理研究所)