下一次星系巡天的标度对计数
Scaling pair count to next galaxy surveys
摘要：
根据距离对星系或恒星对进行计数是天体物理学和宇宙学中实空间相关分析的核心。即将到来的星系巡天（LSST，欧几里得）将测量数十亿个星系的属性，挑战我们在与模拟使用相关的分钟尺度时间内执行此类计数的能力。这个问题只受到有效访问数据的限制，因此属于大数据范畴。我们使用流行的Apache Spark框架来解决这个问题，并设计了一个高效的高吞吐量算法来处理数亿到数十亿的输入数据。为了优化它，我们重新审视了基于立方体对称性的非层次球体像素化问题，并开发了一个新的称为“相似半径球体像素化”（SARSPix）的非常接近方形像素。它为所有与距离相关的计算提供了球体上最适合的索引。使用LSST类快速模拟，我们计算包含一亿到十亿数据点的层析箱上的自相关函数。在每种情况下，我们都在大约2分钟内完成了标准对距离直方图的构建，使用了一种简单的算法，该算法可以在中等数量的节点（16到64）上进行扩展。这说明了这种新技术在天文学领域的潜力，其中数据访问正在成为主要瓶颈。它们可以很容易地适应其他用例，如最近邻居搜索，目录交叉匹配或聚类查找。该软件可从https://github.com/astrolabsoftware/SparkCorr公开获得。
英文摘要：
Counting pairs of galaxies or stars according to their distance is at the core of real-space correlation analyzes performed in astrophysics and cosmology. Upcoming galaxy surveys (LSST, Euclid) will measure properties of billions of galaxies challenging our ability to perform such counting in a minute-scale time relevant for the usage of simulations. The problem is only limited by efficient access to the data, hence belongs to the big data category. We use the popular Apache Spark framework to address it and design an efficient high-throughput algorithm to deal with hundreds of millions to billions of input data. To optimize it, we revisit the question of nonhierarchical sphere pixelization based on cube symmetries and develop a new one dubbed the "Similar Radius Sphere Pixelization" (SARSPix) with very close to square pixels. It provides the most adapted indexing over the sphere for all distance-related computations. Using LSST-like fast simulations, we compute autocorrelation functions on tomographic bins containing between a hundred million to one billion data points. In each case we achieve the construction of a standard pair-distance histogram in about 2 minutes, using a simple algorithm that is shown to scale, over a moderate number of nodes (16 to 64). This illustrates the potential of this new techniques in the field of astronomy where data access is becoming the main bottleneck. They can be easily adapted to other use-cases as nearest-neighbors search, catalog cross-match or cluster finding. The software is publicly available from https://github.com/astrolabsoftware/SparkCorr.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
