IoT应用中流数据流的协同重用
Collaborative Reuse of Streaming Dataflows in IoT Applications
摘要：
分布式流处理系统（DSPS），如Apache Storm和Spark Streaming，可以组成持续在数据流上执行的连续流。物联网（IoT）应用程序使用它们来分析来自智能城市网络基础设施的传感器数据，并做出积极的公用事业管理决策。随着利用共享城市传感器流的物联网应用生态系统的不断发展，应用程序将执行重复的预处理和分析任务。这提供了协作重用重叠流程的产出的机会，从而提高了资源效率。在本文中，我们提出了\emph{dataflow reuse algorithms}，给定一个提交的流，识别可重用的任务和流的交集从一个集合的运行流，形成一个\emph{merged dataflow}。还提出了类似的算法，以unmerge的波纹时，他们被删除。我们为流行的Apache Storm DSPS实现了这些算法，并基于具有不同到达和离开分布的公共OPMW工作流以及来自RIoTBench的21个真实物联网路由器，验证了35个合成路由器的性能和资源节省。
英文摘要：
Distributed Stream Processing Systems (DSPS) like Apache Storm and Spark Streaming enable composition of continuous dataflows that execute persistently over data streams. They are used by Internet of Things (IoT) applications to analyze sensor data from Smart City cyber-infrastructure, and make active utility management decisions. As the ecosystem of such IoT applications that leverage shared urban sensor streams continue to grow, applications will perform duplicate pre-processing and analytics tasks. This offers the opportunity to collaboratively reuse the outputs of overlapping dataflows, thereby improving the resource efficiency. In this paper, we propose \emph{dataflow reuse algorithms} that given a submitted dataflow, identifies the intersection of reusable tasks and streams from a collection of running dataflows to form a \emph{merged dataflow}. Similar algorithms to unmerge dataflows when they are removed are also proposed. We implement these algorithms for the popular Apache Storm DSPS, and validate their performance and resource savings for 35 synthetic dataflows based on public OPMW workflows with diverse arrival and departure distributions, and on 21 real IoT dataflows from RIoTBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
