SDSS批量查询系统在国家虚拟天文台网格中的扩展
Extending the SDSS Batch Query System to the National Virtual Observatory Grid
摘要：
斯隆数字巡天科学数据库接近2 TB。虽然绝大多数查询通常在几秒或几分钟内执行，但这种交互式执行时间可能会因一小部分需要几小时或几天运行的查询而不成比例地增加;因为它们需要对最大的表进行非索引扫描，或者因为它们请求非常大的结果集。针对这一点，我们添加了一个多队列作业提交和跟踪系统。通过网络传输查询的非常大的结果集是另一个严重的问题。统计数据表明，这种数据传输大部分是不必要的;用户更愿意将结果存储在本地，以便进一步交叉匹配和过滤。为了允许本地分析，我们实现了一个系统，在门户网站上为用户提供他们自己的个人数据库（MyDB）。用户可以将数据传输到他们的MyDB，然后在将其提取到自己的机器之前进行进一步分析。我们打算将MyDB和异步查询思想扩展到多个NVO节点。这意味着以分布式方式开发若干功能，这些功能已在SDSS批量查询系统（CasJobs）的单个节点上得到演示。异步查询的泛化需要某种形式的MyDB存储以及每个节点上的工作流跟踪服务和节点之间的协调策略。
英文摘要：
The Sloan Digital Sky Survey science database is approaching 2TB. While the vast majority of queries normally execute in seconds or minutes, this interactive execution time can be disproportionately increased by a small fraction of queries that take hours or days to run; either because they require non-index scans of the largest tables or because they request very large result sets. In response to this, we added a multi-queue job submission and tracking system. The transfer of very large result sets from queries over the network is another serious problem. Statistics suggested that much of this data transfer is unnecessary; users would prefer to store results locally in order to allow further cross matching and filtering. To allow local analysis, we implemented a system that gives users their own personal database (MyDB) at the portal site. Users may transfer data to their MyDB, and then perform further analysis before extracting it to their own machine. We intend to extend the MyDB and asynchronous query ideas to multiple NVO nodes. This implies development, in a distributed manner, of several features, which have been demonstrated for a single node in the SDSS Batch Query System (CasJobs). The generalization of asynchronous queries necessitates some form of MyDB storage as well as workflow tracking services on each node and coordination strategies among nodes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
