jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models

New York University（纽约大学）；Google Research（谷歌研究院）

Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, Samuel R. Bowman

英文摘要：

We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. jiant enables modular and configuration-driven experimentation with state-of-the-art models and implements a broad set of tasks for probing, transfer learning, and multitask training experiments. jiant implements over 50 NLU tasks, including all GLUE and SuperGLUE benchmark tasks. We demonstrate that jiant reproduces published performance on a variety of tasks and models, including BERT and RoBERTa. jiant is available at https://jiant.info.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Russian SuperGLUE 1.1: Revising the Lessons not Learned by Russian NLP models

SberDevices；Sberbank（俄罗斯联邦储蓄银行）；National Research University Higher School of Economics（国立研究型高等经济学院）；ANO «AI Research Institute»（“人工智能研究院”自治非营利组织）；Alex Kukushkin Lab（Alex Kukushkin实验室）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Huawei Noah’s Ark lab（华为诺亚方舟实验室）；Kazan Federal University（喀山联邦大学）

In the last year, new neural architectures and multilingual pre-trained models have been released for Russian, which led to performance evaluation problems across a range of language understanding tasks. This paper presents Russian SuperGLUE 1.1, an updated benchmark styled after GLUE for Russian NLP models. The new version includes a number of technical, user experience and methodological improvements, including fixes of the benchmark vulnerabilities unresolved in the previous version: novel and improved tests for understanding the meaning of a word in context (RUSSE) along with reading compr…

2022-02-15cs.AIcs.CL

UER: An Open-Source Toolkit for Pre-training Models

Renmin University of China（中国人民大学）；Tencent AI Lab（腾讯人工智能实验室）；Peking University（北京大学）

Existing works, including ELMO and BERT, have revealed the importance of pre-training for NLP tasks. While there does not exist a single pre-training model that works best in all cases, it is of necessity to develop a framework that is able to deploy various pre-training models efficiently. For this purpose, we propose an assemble-on-demand pre-training toolkit, namely Universal Encoder Representations (UER). UER is loosely coupled, and encapsulated with rich modules. By assembling modules on demand, users can either reproduce a state-of-the-art pre-training model or develop a pre-training mod…

2019-09-12cs.CLcs.LG

SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

New York University（纽约大学）；Facebook AI Research（Facebook人工智能研究）；University of Washington（华盛顿大学）；DeepMind（深度思维）

In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks. The GLUE benchmark, introduced a little over one year ago, offers a single-number metric that summarizes progress on a diverse set of such tasks, but performance on the benchmark has recently surpassed the level of non-expert humans, suggesting limited headroom for further research. In this paper we present SuperGLUE, a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, a software…

2020-02-13cs.AIcs.CL

SanskritShala：一个用于教学与标注目的的、具有网页界面的神经梵语自然语言处理工具包

SanskritShala: A Neural Sanskrit NLP Toolkit with Web-Based Interface for Pedagogical and Annotation Purposes

IIT Kanpur（印度理工学院坎普尔分校）；IIT Kharagpur（印度理工学院克勒格布尔分校）；IIT Mandi（印度理工学院曼迪分校）

本文提出首个带网页界面的神经梵语NLP工具包SanskritShala，集成词切分、形态标注、依存解析和复合词识别等模块，在多个基准上达到最先进性能，并公开代码、词嵌入模型及标注数据，以支持梵语教学与标注工作。

2023-05-29cs.CL

calamanCy：一个他加禄语自然语言处理工具包

calamanCy: A Tagalog Natural Language Processing Toolkit

ExplosionAI GmbH（ExplosionAI 有限公司）

calamanCy是一个基于spaCy的开源他加禄语NLP工具包，提供统一API和多任务模型，支持依存句法分析、词性标注和命名实体识别，以加速该语言NLP发展。

2023-11-13cs.CL

XNMT: The eXtensible Neural Machine Translation Toolkit

Carnegie Mellon University（卡内基梅隆大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；LIMSI/CNRS/Université Paris-Saclay（巴黎萨克雷大学LIMSI/法国国家科学研究中心）；University of Pennsylvania（宾夕法尼亚大学）；ENS/CNRS/EHESS/INRIA（巴黎高等师范学院/法国国家科学研究中心/巴黎高等经济社会学院/法国国家信息与自动化研究所）；University of Illinois（伊利诺伊大学）

This paper describes XNMT, the eXtensible Neural Machine Translation toolkit. XNMT distin- guishes itself from other open-source NMT toolkits by its focus on modular code design, with the purpose of enabling fast iteration in research and replicable, reliable results. In this paper we describe the design of XNMT and its experiment configuration system, and demonstrate its utility on the tasks of machine translation, speech recognition, and multi-tasked machine translation/parsing. XNMT is available open-source at https://github.com/neulab/xnmt

2018-03-01cs.CL

更多相关论文收起更多论文