arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1705.10754cs.CL
收藏

一种用于语言变体识别的低模糊度表示方法

A Low Dimensionality Representation for Language Variety Identification

Autoritas Consulting(Autoritas咨询公司);Universitat Politècnica de València(瓦伦西亚理工大学)

Francisco Rangel, Marc Franco-Salvador, Paolo Rosso

摘要:

语言种类识别的目的是用母语(例如西班牙语、葡萄牙语、英语)标记文本及其特定变体(例如阿根廷、智利、墨西哥、秘鲁、西班牙;巴西、葡萄牙;英国、美国)。在这项工作中,我们提出了一个低维表示(LDR)来解决这个任务与五个不同品种的西班牙语:阿根廷,智利,墨西哥,秘鲁和西班牙。我们将我们的LDR方法与常见的最先进的表示方法进行了比较,结果显示精度提高了约35%。此外,我们比较LDR与两个参考分布式表示模型。实验结果显示了有竞争力的性能,同时大大降低了维度-并增加了大数据的适用性-每个品种只有6个特征。此外,我们分析了所采用的机器学习算法的行为和最具鉴别力的特征。最后,我们采用另一种数据集来测试我们的低维表示与另一组类似语言的鲁棒性。

英文摘要:

Language variety identification aims at labelling texts in a native language (e.g. Spanish, Portuguese, English) with its specific variation (e.g. Argentina, Chile, Mexico, Peru, Spain; Brazil, Portugal; UK, US). In this work we propose a low dimensionality representation (LDR) to address this task with five different varieties of Spanish: Argentina, Chile, Mexico, Peru and Spain. We compare our LDR method with common state-of-the-art representations and show an increase in accuracy of ~35%. Furthermore, we compare LDR with two reference distributed representation models. Experimental results show competitive performance while dramatically reducing the dimensionality --and increasing the big data suitability-- to only 6 features per variety. Additionally, we analyse the behaviour of the employed machine learning algorithms and the most discriminating features. Finally, we employ an alternative dataset to test the robustness of our low dimensionality representation with another set of similar languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑