使用专用网络和并行网络协同构建经济高效的分布式流处理平台
Using Dedicated and Opportunistic Networks in Synergy for a Cost-effective Distributed Stream Processing Platform
摘要：
本文提出了一个案例，利用专用和机会网络资源的协同作用，在分布式托管平台的数据流处理应用程序。我们以前的研究已经证明了在高吞吐量计算应用程序中，及时分配处理单元是主要关注点的情况下，将专用可靠资源与机会资源相结合的好处。由于分布式流处理应用程序需要以一致的速率在处理站点之间进行大量数据传输，因此对网络资源的适当控制对于确保稳定的处理流非常重要。在本文中，我们提出了一个系统模型的混合托管平台，其中安装在分布式站点的流处理服务器互连的专用链接和公共互联网的组合。分散式算法已经被开发用于在竞争任务之间分配两类网络资源，其目标是实现更高的任务吞吐量和更好地利用昂贵的专用资源。从广泛的模拟研究结果表明，适当的管理，系统利用专用和机会资源的协同作用，产生相当高的任务吞吐量，因此，更高的投资回报率比系统单独使用昂贵的专用资源。
英文摘要：
This paper presents a case for exploiting the synergy of dedicated and opportunistic network resources in a distributed hosting platform for data stream processing applications. Our previous studies have demonstrated the benefits of combining dedicated reliable resources with opportunistic resources in case of high-throughput computing applications, where timely allocation of the processing units is the primary concern. Since distributed stream processing applications demand large volume of data transmission between the processing sites at a consistent rate, adequate control over the network resources is important here to assure a steady flow of processing. In this paper, we propose a system model for the hybrid hosting platform where stream processing servers installed at distributed sites are interconnected with a combination of dedicated links and public Internet. Decentralized algorithms have been developed for allocation of the two classes of network resources among the competing tasks with an objective towards higher task throughput and better utilization of expensive dedicated resources. Results from extensive simulation study show that with proper management, systems exploiting the synergy of dedicated and opportunistic resources yield considerably higher task throughput and thus, higher return on investment over the systems solely using expensive dedicated resources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
