用MapReduce对印度尼西亚巴丹岛进行最大熵POS标记的简化
Parallelization of Maximum Entropy POS Tagging for Bahasa Indonesia with MapReduce
摘要:
本文采用MapReduce编程模型对印度尼西亚语最大熵词性标注的训练和标注过程进行并行化。在训练过程中,MapReduce模型实现了字典、标记和特征创建。在标记过程中,MapReduce实现了并行标记文档行。训练实验表明,使用MapReduce的总训练时间更快,但其过程中的结果读取时间减慢了总训练时间。使用不同数量的map和reduce过程进行标注实验,结果表明MapReduce实现可以加快标注过程。通过对100万词语料库和30个图的标注过程,显示出最快的标注结果。
英文摘要:
In this paper, MapReduce programming model is used to parallelize training and tagging proceess in Maximum Entropy part of speech tagging for Bahasa Indonesia. In training process, MapReduce model is implemented dictionary, tagtoken, and feature creation. In tagging process, MapReduce is implemented to tag lines of document in parallel. The training experiments showed that total training time using MapReduce is faster, but its result reading time inside the process slow down the total training time. The tagging experiments using different number of map and reduce process showed that MapReduce implementation could speedup the tagging process. The fastest tagging result is showed by tagging process using 1,000,000 word corpus and 30 map process.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。