arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.03412cs.CL
收藏

语言处理的词典和语法:工业产品还是手工制品?

Lexicons and grammars for language processing: industrial or handcrafted products?

Université Paris-Est(巴黎东部大学);Universidade Estadual Paulista (UNESP)(圣保罗州立大学);University of Belgrade(贝尔格莱德大学)

Eric Laporte

AI总结:

本文分析词典和语法的手工构建与自动化工业化两种趋势,探讨哪种方式或两者结合能获得最佳结果。

摘要:

近年来,语言数据在语言处理中的应用逐渐增加。这些数据现在通常被称为语言资源。用于此目的的大多数语言资源是文本集合,如布朗语料库和宾州树库,但电子词典(WordNet、FrameNet、VerbNet、ComLex、词典-语法...)和形式语法(TAG...)也在最近得到发展。词典和语法的大多数构建过程是手动的,而语料库的构建则一直高度自动化。然而,越来越多的语言处理专家认识到,词典和语法的信息内容比语料库更丰富,因此前者可以实现更精细的处理。构建时间的差异可能与信息内容的差异有关:语言学家手工制作词典和语法可能使其比自动生成的数据更具信息性。这种情况可能向两个方向发展:要么语言技术专家逐渐习惯于处理手工构建的资源,这些资源更具信息性且更复杂;要么词典和语法的构建过程被自动化和工业化,这是主流观点。两种演变都在进行中,并且它们之间存在紧张关系。语言学家和计算机科学家之间的关系取决于这些演变的未来,因为前者需要培训和雇佣大量语言学家,而后者主要依赖于计算机工程师提出的解决方案。本文旨在分析所讨论的语言资源的实际例子,并讨论手工制作或工业生成,或两者结合,哪种趋势能产生最佳结果或最现实。

英文摘要:

During the recent years, the use of linguistic data for language processing increased progressively. Such data are now commonly called language resources. Most of the language resources used for this purpose are collections of texts as the Brown Corpus and the Penn Treebank, but electronic lexicons (WordNet, FrameNet, VerbNet, ComLex, Lexicon-Grammar...) and formal grammars (TAG...) developed recently. Most processes of construction of lexicons and grammars are manual, whereas the construction of corpora has always been highly automated. However, more and more specialists of language processing realize that the information content of lexicons and grammars is richer than that of corpora, and hence the former make more elaborate processing possible. The difference in construction time is likely to be connected with the difference in information content: the handcrafting of lexicons and grammars by linguists would make them more informative than automatically generated data. This situation can evolve into two directions: either specialists of language technology get progressively used to handling manually constructed resources, which are more informative and more complex, or the process of construction of lexicons and grammars is automated and industrialized, which is the mainstream perspective. Both evolutions are already in progress, and a tension exists between them. The relation between linguists and computer scientists depends on the future of these evolutions, since the first implies training and hiring numerous linguists, whereas the other depends essentially on solutions elaborated by computer engineers. The aim of this article is to analyse practical examples of the language resources in question, and to discuss about which of the two trends, handcrafting or generating industrially, or a combination of both, can give the best results or is the most realistic.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑