PyPLN:一个分布式自然语言处理平台
PyPLN: a Distributed Platform for Natural Language Processing
摘要:
本文提出了一个分布式自然语言处理平台PyPLN。PyPLN利用大量的NLP和文本处理开源工具,管理各种配置上的工作负载分布:从单个服务器到Linux服务器集群。PyPLN是使用Python 2.7.3开发的,但只要Linux版本可用,就可以很容易地将其他软件用于特定任务。PyPLN促进了文档和语料库级别的分析,通过易于使用的Web界面简化了语料库和分析结果的管理和发布。在当前(beta)版本中,它支持英语和葡萄牙语,并支持计划在未来版本中使用的其他语言。为了支持葡萄牙语,PyPLN使用PALAVRAS解析器\citep{Bick2000}。目前PyPLN提供以下功能:编码规范化的文本提取(到UTF-8),词性标记,标记频率,语义注释,n-gram提取,单词和句子库以及跨语料库的全文搜索。该平台以GPL-v3许可。
英文摘要:
This paper presents a distributed platform for Natural Language Processing called PyPLN. PyPLN leverages a vast array of NLP and text processing open source tools, managing the distribution of the workload on a variety of configurations: from a single server to a cluster of linux servers. PyPLN is developed using Python 2.7.3 but makes it very easy to incorporate other softwares for specific tasks as long as a linux version is available. PyPLN facilitates analyses both at document and corpus level, simplifying management and publication of corpora and analytical results through an easy to use web interface. In the current (beta) release, it supports English and Portuguese languages with support to other languages planned for future releases. To support the Portuguese language PyPLN uses the PALAVRAS parser\citep{Bick2000}. Currently PyPLN offers the following features: Text extraction with encoding normalization (to UTF-8), part-of-speech tagging, token frequency, semantic annotation, n-gram extraction, word and sentence repertoire, and full-text search across corpora. The platform is licensed as GPL-v3.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。