PB级数据采掘：梦想还是现实？
Petabyte Scale Data Mining: Dream or Reality?
摘要：
科学正在变得非常数据密集1。今天的天文学数据集与数以千万计的星系已经提出了巨大的挑战数据采掘。在不到10年的时间里，目录预计将增长到数十亿个对象，图像存档将达到PB。想象一下，在1996年有一个100 GB的数据库，当时磁盘扫描速度为30 MB/s，数据库工具还不成熟。今天这样的任务是微不足道的，几乎可以用笔记本电脑来管理。我们认为PB数据库的问题在六年内将非常相似。在本文中，我们规模我们目前的实验数据存档和分析的斯隆数字巡天2，3数据六年的未来。我们分析这些预测，并在这些数据集上执行数据采掘的要求。我们的结论是，这项任务的规模相当不错：我们今天就可以完成这项工作，尽管代价高昂。似乎没有任何阻止我们存储和使用六年后的PB数据集的障碍。
英文摘要：
Science is becoming very data intensive1. Today's astronomy datasets with tens of millions of galaxies already present substantial challenges for data mining. In less than 10 years the catalogs are expected to grow to billions of objects, and image archives will reach Petabytes. Imagine having a 100GB database in 1996, when disk scanning speeds were 30MB/s, and database tools were immature. Such a task today is trivial, almost manageable with a laptop. We think that the issue of a PB database will be very similar in six years. In this paper we scale our current experiments in data archiving and analysis on the Sloan Digital Sky Survey2,3 data six years into the future. We analyze these projections and look at the requirements of performing data mining on such data sets. We conclude that the task scales rather well: we could do the job today, although it would be expensive. There do not seem to be any show-stoppers that would prevent us from storing and using a Petabyte dataset six years from today.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
