鲁棒机器学习在Terascale天文数据集上的应用
Robust Machine Learning Applied to Terascale Astronomical Datasets
摘要:
我们介绍了国家超级计算应用中心(NCSA)宇宙学数据挖掘实验室(http://www.example.com)的最新结果,为万亿级斯隆数字巡天(SDSS)中的天体提供了可靠的分类和光度红移。lcdm.astro.uiuc.edu通过决策树、k-近邻和遗传算法形式的机器学习、NCSA超级计算资源的使用以及网络增强数据到知识的结合,我们能够为SDSS中的1亿多个对象提供改进的分类、改进的光度红移以及充分利用强大的k-近邻算法。这项工作是第一次将这些算法的全部功能应用于当代太尺度天文数据集,并且对现有结果的改进是显而易见的。我们讨论了我们在处理万亿次数据时遇到的问题,以及可以实现来处理即将到来的千万亿次数据集的可能解决方案。
英文摘要:
We present recent results from the Laboratory for Cosmological Data Mining (http://lcdm.astro.uiuc.edu) at the National Center for Supercomputing Applications (NCSA) to provide robust classifications and photometric redshifts for objects in the terascale-class Sloan Digital Sky Survey (SDSS). Through a combination of machine learning in the form of decision trees, k-nearest neighbor, and genetic algorithms, the use of supercomputing resources at NCSA, and the cyberenvironment Data-to-Knowledge, we are able to provide improved classifications for over 100 million objects in the SDSS, improved photometric redshifts, and a full exploitation of the powerful k-nearest neighbor algorithm. This work is the first to apply the full power of these algorithms to contemporary terascale astronomical datasets, and the improvement over existing results is demonstrable. We discuss issues that we have encountered in dealing with data on the terascale, and possible solutions that can be implemented to deal with upcoming petascale datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。