BlueOx:一个用于分布式数据分析的Java框架
BlueOx: A Java Framework for Distributed Data Analysis
摘要:
包括Tevatron和即将到来的LHC在内的高能物理实验需要分析大型数据集,这些数据集最好由分布式计算处理。介绍了一个基于Java的分布式数据分析框架的设计与开发。分析作业通过三个阶段运行:发现可用的数据集,将数据集代理/分配给分析服务器,以及作业执行。每个阶段由一组抽象接口表示。这些接口允许使用不同的技术而不修改框架。例如,通信接口已经通过分组协议和基于SOAP的方案两者来实现。用户认证可以通过简单的密码或通过GSI证书系统提供。来自CMS HCAL Testbeams,L3 LEP实验和假设的高能线性对撞机实验的数据已经与框架接口。
英文摘要:
High energy physics experiments including those at the Tevatron and the upcoming LHC require analysis of large data sets which are best handled by distributed computation. We present the design and development of a distributed data analysis framework based on Java. Analysis jobs run through three phases: discovery of data sets available, brokering/assignment of data sets to analysis servers, and job execution. Each phase is represented by a set of abstract interfaces. These interfaces allow different techniques to be used without modification to the framework. For example, the communications interface has been implemented by both a packet protocol and a SOAP-based scheme. User authentication can be provided either through simple passwords or through a GSI certificates system. Data from CMS HCAL Testbeams, the L3 LEP experiment, and a hypothetical high-energy linear collider experiment have been interfaced with the framework.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。