SuperGLUE：通用语言理解系统的一个比较敏感的基准
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
摘要：
在过去的一年里，预训练和迁移学习的新模型和方法在一系列语言理解任务中取得了显着的成绩提升。一年多前推出的GLUE基准测试提供了一个单一的数字指标，总结了各种此类任务的进展，但该基准测试的表现最近已经超过了非专家人类的水平，这表明进一步研究的空间有限。在本文中，我们提出了SuperGLUE，一个新的基准风格后，GLUE与一组新的更困难的语言理解任务，一个软件工具包，和一个公共排行榜。SuperGLUE可在super.gluebenchmark.com上获得。
英文摘要：
In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks. The GLUE benchmark, introduced a little over one year ago, offers a single-number metric that summarizes progress on a diverse set of such tasks, but performance on the benchmark has recently surpassed the level of non-expert humans, suggesting limited headroom for further research. In this paper we present SuperGLUE, a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, a software toolkit, and a public leaderboard. SuperGLUE is available at super.gluebenchmark.com.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
