工程众包流处理系统
Engineering Crowdsourced Stream Processing Systems
摘要:
众包流处理系统(CSP)是在数据流的处理中并入众包任务的系统。这可以被视为使众包工作能够高速应用于大规模数据样本,或者等同地,使流处理能够利用人类智能。它还导致数据处理系统能力的大幅扩展。CSP系统的工程设计需要人机计算元素的结合。从一般系统理论的角度来看,这意味着要考虑这两个元素的继承和新兴属性。在本文中,我们定位CSP系统在一个更广泛的分类,概述了一系列的设计原则和评估指标,提出了一个可扩展的框架,他们的设计,并描述了几种设计模式。我们展示了CSP系统的能力,通过执行一个案例研究,将我们提出的框架应用于一个真实的系统(AIDR)的设计和分析,该系统在时间关键的危机事件中对社交媒体消息进行分类。结果表明,与纯流处理系统相比,AIDR可以实现更高的数据分类精度,而与纯众包解决方案相比,该系统通过需要更少的人工工作量来更好地利用人类工作者。
英文摘要:
A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream. This can be seen as enabling crowdsourcing work to be applied on a sample of large-scale data at high speed, or equivalently, enabling stream processing to employ human intelligence. It also leads to a substantial expansion of the capabilities of data processing systems. Engineering a CSP system requires the combination of human and machine computation elements. From a general systems theory perspective, this means taking into account inherited as well as emerging properties from both these elements. In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design patterns. We showcase the capabilities of CSP systems by performing a case study that applies our proposed framework to the design and analysis of a real system (AIDR) that classifies social media messages during time-critical crisis events. Results show that compared to a pure stream processing system, AIDR can achieve a higher data classification accuracy, while compared to a pure crowdsourcing solution, the system makes better use of human workers by requiring much less manual work effort.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。