arXiv 2501.062492025-01cs.CVastro-ph.IM

可扩展的宇宙人工智能推理：基于云无服务器计算

Scalable Cosmic AI Inference using Cloud Serverless Computing

Department of Computer Science, University of Virginia（弗吉尼亚大学计算机科学系）；School of Data Science, University of Virginia（弗吉尼亚大学数据科学学院）

Mills Staylor, Amirreza Dolatpour Fathkouhi, Md Khairul Islam, Kaleigh O'Hara, Ryan Ghiles Goudjil, Geoffrey Fox, Judy Fox

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出基于云无服务器计算的CAI框架，结合预训练模型与FaaS实现高效可扩展的天文图像推理，红移预测在28秒内完成12.6GB数据处理，成本低于5美元，支持TB级数据。

摘要：

大规模天文图像数据处理与预测对天文学家至关重要，可为天体、宇宙历史及其演化提供关键见解。尽管现代深度学习模型具有较高的预测精度，但它们通常需要大量的计算资源，导致资源密集且可及性受限。我们提出了基于云的天文推理（CAI）框架以应对这些挑战。这一可扩展解决方案通过函数即服务（FaaS）将预训练基础模型与无服务器云基础设施相结合。CAI能够在无需大量硬件的情况下对天文图像进行高效且可扩展的推理。以用于红移预测的基础模型作为案例研究，我们进行了涵盖用户设备、高性能计算（HPC）服务器和云端的广泛实验。使用AstroMAE模型进行红移预测展示了CAI的可扩展性和效率，在仅28秒内完成了对12.6 GB数据集的推理，而HPC GPU需要140.8秒，HPC CPU需要1793秒。CAI还实现了显著更高的吞吐量，达到每秒180.4亿比特（bps），并在数据规模增加时保持近乎恒定的推理时间，且计算成本极低（每次实验低于5美元）。我们还处理了高达1 TB的大规模数据，以展示CAI在大规模场景下的有效性。因此，CAI为天文学界提供了一种高度可扩展、可访问且成本效益高的推理解决方案。代码可在https://github.com/UVA-MLSys/AI-for-Astronomy获取。

英文摘要：

Large-scale astronomical image data processing and prediction are essential for astronomers, providing crucial insights into celestial objects, the universe's history, and its evolution. While modern deep learning models offer high predictive accuracy, they often demand substantial computational resources, making them resource-intensive and limiting accessibility. We introduce the Cloud-based Astronomy Inference (CAI) framework to address these challenges. This scalable solution integrates pre-trained foundation models with serverless cloud infrastructure through a Function-as-a-Service (FaaS). CAI enables efficient and scalable inference on astronomical images without extensive hardware. Using a foundation model for redshift prediction as a case study, our extensive experiments cover user devices, HPC (High-Performance Computing) servers, and Cloud. Using redshift prediction with the AstroMAE model demonstrated CAI's scalability and efficiency, achieving inference on a 12.6 GB dataset in only 28 seconds compared to 140.8 seconds on HPC GPUs and 1793 seconds on HPC CPUs. CAI also achieved significantly higher throughput, reaching 18.04 billion bits per second (bps), and maintained near-constant inference times as data sizes increased, all at minimal computational cost (under $5 per experiment). We also process large-scale data up to 1 TB to show CAI's effectiveness at scale. CAI thus provides a highly scalable, accessible, and cost-effective inference solution for the astronomy community. The code is accessible at https://github.com/UVA-MLSys/AI-for-Astronomy.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。