测试MapReduce-Based系统
Testing MapReduce-Based Systems
摘要：
MapReduce（MR）是构建大规模数据处理应用程序的最流行的解决方案。这些应用程序通常部署在大型商用机器集群上，由于错误、硬件问题和停机，这些集群经常发生故障。基于MR的系统的测试是困难的，因为它需要大量的工作，测试线束执行分布式测试用例失败。在本文中，我们提出了一种新的测试解决方案来解决这个问题，称为Hadoop测试。该解决方案基于可扩展的线束方法，其中分布式测试器组件挂在每个映射周围并减少工作器（即，节点）。测试人员可以激励每个工作人员在他们身上注入故障，监控他们的行为，并验证测试结果。Hadoop Test用于测试捆绑到Hadoop中的两个应用程序，Hadoop是Apache开源MapReduce实现。我们的初始实现展示了有希望的结果，Hadoop测试在分布式MapReduce工作者之间协调测试用例，并发现错误。
英文摘要：
MapReduce (MR) is the most popular solution to build applications for large-scale data processing. These applications are often deployed on large clusters of commodity machines, where failures happen constantly due to bugs, hardware problems, and outages. Testing MR-based systems is hard, since it is needed a great effort of test harness to execute distributed test cases upon failures. In this paper, we present a novel testing solution to tackle this issue called HadoopTest. This solution is based on a scalable harness approach, where distributed tester components are hung around each map and reduce worker (i.e., node). Testers are allowed to stimulate each worker to inject failures on them, monitor their behavior, and validate testing results. HadoopTest was used to test two applications bundled into Hadoop, the Apache open source MapReduce implementation. Our initial implementation demonstrates promising results, with HadoopTest coordinating test cases across distributed MapReduce workers, and finding bugs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
