arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.11443cs.CL
收藏

语言领域数据管理与分析系统-- LiFE演示

Demo of the Linguistic Field Data Management and Analysis System -- LiFE

Dr. Bhimrao Ambedkar University(宾劳·安贝德卡尔大学)

Siddharth Singh, Ritesh Kumar, Shyam Ratan, Sonal Sinha

摘要:

在拟议的演示中,我们将介绍一个新的软件-语言领域数据管理和分析系统- LiFE(https://github.com/kmi-linguistics/life)-一个开源的,基于Web的语言数据管理和分析应用程序,允许系统存储,管理,共享和使用从现场收集的语言数据。该应用程序允许用户存储词汇项,句子,段落,具有丰富注释/注释的视听内容;生成交互式和打印词典;还可以使用这些数据为各种目的进行训练和使用自然语言处理工具和模型。由于它是一个基于Web的应用程序,它还允许多人之间的无缝协作,并相互共享数据,模型等。该系统在后端使用基于Python的Flask框架和MongoDB,在前端使用HTML,CSS和JavaScript。该界面允许创建多个项目,可以与其他用户共享。在后端,应用程序以RDF格式存储数据,以便使用语义Web技术将其作为链接数据发布到Web上-到目前为止,它使用OntoLex-Lemon存储词汇数据,并使用Ligt存储线间注释文本,然后将其内部链接到其他链接的词典和数据库,如DBpedia和WordNet。此外,它还支持使用scikit-learn和HuggingFace Transformers库训练NLP系统,并利用使用这些库训练的任何模型-虽然用户界面本身提供了有限的调整系统的选项,但外部训练的模型可以很容易地合并到应用程序中;类似地,数据集本身可以很容易地导出为标准的机器可读格式,如JSON或CSV,可以由其他程序和管道使用。

英文摘要:

In the proposed demo, we will present a new software - Linguistic Field Data Management and Analysis System - LiFE (https://github.com/kmi-linguistics/life) - an open-source, web-based linguistic data management and analysis application that allows for systematic storage, management, sharing and usage of linguistic data collected from the field. The application allows users to store lexical items, sentences, paragraphs, audio-visual content with rich glossing / annotation; generate interactive and print dictionaries; and also train and use natural language processing tools and models for various purposes using this data. Since its a web-based application, it also allows for seamless collaboration among multiple persons and sharing the data, models, etc with each other. The system uses the Python-based Flask framework and MongoDB in the backend and HTML, CSS and Javascript at the frontend. The interface allows creation of multiple projects that could be shared with the other users. At the backend, the application stores the data in RDF format so as to allow its release as Linked Data over the web using semantic web technologies - as of now it makes use of the OntoLex-Lemon for storing the lexical data and Ligt for storing the interlinear glossed text and then internally linking it to the other linked lexicons and databases such as DBpedia and WordNet. Furthermore it provides support for training the NLP systems using scikit-learn and HuggingFace Transformers libraries as well as make use of any model trained using these libraries - while the user interface itself provides limited options for tuning the system, an externally-trained model could be easily incorporated within the application; similarly the dataset itself could be easily exported into a standard machine-readable format like JSON or CSV that could be consumed by other programs and pipelines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑