arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1303.1932cs.CL
收藏

PANACEA平台中领域语料库的挖掘与开发

Mining and Exploiting Domain-Specific Corpora in the PANACEA Platform

Universitat Pompeu Fabra(庞培法布拉大学);Institute for Language and Speech Processing/Athena RIC(语言与语音处理研究所/雅典娜研究中心);Dublin City University(都柏林城市大学);ELDA(评估与语言资源分发协会)

Núria Bel, Vassilis Papavasiliou, Prokopis Prokopidis, Antonio Toral, Victoria Arranz

摘要:

PANACEA ICT-2007.2.2 EU项目的目标是建立一个平台,使MT系统所需的大型语言资源的获取、生产、更新和维护等阶段实现自动化。开发用于从网络中提取单语和双语数据的语料库获取组件(CAC)是PANACEA最具创新性的构建模块之一。CAC是PANACEA管道中用于构建语言资源的第一阶段,采用高效的分布式方法来抓取特定语言和预定义域中具有丰富文本内容的Web文档。CAC包括一些模块,这些模块可以从具有多种语言的域内内容的网站获取并行数据。为了对CAC方法进行进一步的评估,我们进行了几个实验,使用爬行的平行语料库,通过句子对齐来识别和提取平行句子。该语料库被成功地用于机器翻译系统的领域自适应。

英文摘要:

The objective of the PANACEA ICT-2007.2.2 EU project is to build a platform that automates the stages involved in the acquisition, production, updating and maintenance of the large language resources required by, among others, MT systems. The development of a Corpus Acquisition Component (CAC) for extracting monolingual and bilingual data from the web is one of the most innovative building blocks of PANACEA. The CAC, which is the first stage in the PANACEA pipeline for building Language Resources, adopts an efficient and distributed methodology to crawl for web documents with rich textual content in specific languages and predefined domains. The CAC includes modules that can acquire parallel data from sites with in-domain content available in more than one language. In order to extrinsically evaluate the CAC methodology, we have conducted several experiments that used crawled parallel corpora for the identification and extraction of parallel sentences using sentence alignment. The corpora were then successfully used for domain adaptation of Machine Translation Systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑