RIOT:无SQL的I/O高效数值计算
RIOT: I/O-Efficient Numerical Computing without SQL
摘要:
R是一种数值计算环境,广泛用于统计数据分析。像许多这样的环境一样,R对于大小超过物理内存的大型数据集表现不佳。我们提出了我们的RIOT(R与I/O透明),一个系统,使R程序的I/O效率的方式透明的用户。我们描述我们的经验与RIOT-DB,一个初始的原型,使用关系数据库系统作为后端。尽管通用数据库系统在处理数组数据和数值计算方面存在开销和不足,但RIOT-DB在许多大数据场景中的性能明显优于R,这要归功于一套无缝集成到R中的高级互操作优化。虽然RIOT中的许多技术都受到数据库的启发(对于RIOT-DB来说,是由数据库系统实现的),但RIOT用户与数据库相关的任何东西都是绝缘的。与以前需要用户学习新语言并重写程序以与数据库接口的方法相比,我们相信RIOT将更容易被大多数R用户采用。
英文摘要:
R is a numerical computing environment that is widely popular for statistical data analysis. Like many such environments, R performs poorly for large datasets whose sizes exceed that of physical memory. We present our vision of RIOT (R with I/O Transparency), a system that makes R programs I/O-efficient in a way transparent to the users. We describe our experience with RIOT-DB, an initial prototype that uses a relational database system as a backend. Despite the overhead and inadequacy of generic database systems in handling array data and numerical computation, RIOT-DB significantly outperforms R in many large-data scenarios, thanks to a suite of high-level, inter-operation optimizations that integrate seamlessly into R. While many techniques in RIOT are inspired by databases (and, for RIOT-DB, realized by a database system), RIOT users are insulated from anything database related. Compared with previous approaches that require users to learn new languages and rewrite their programs to interface with a database, RIOT will, we believe, be easier to adopt by the majority of the R users.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。