arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.02593cs.CLcs.AI
收藏

金刚鹦鹉通用问答

General-Purpose Question-Answering with Macaw

Allen Institute for Artificial Intelligence(艾伦人工智能研究所)

Oyvind Tafjord, Peter Clark

摘要:

尽管预训练的语言模型取得了成功,但仍然很少有免费的高质量,通用的QA系统。作为回应,我们提出了Macaw,一个多功能的,生成的问答(QA)系统,我们正在向社会提供。Macaw基于UnifiedQA构建,而UnifiedQA本身又基于T5构建,在各种主题上表现出强大的性能(zero-shot),包括在包含300个挑战性问题的测试题集上超过GPT-3 10%(绝对值),尽管其数量级较小(110亿与1750亿参数)。此外,Macaw允许使用其输入和输出的不同排列(“角度”),例如Macaw可以采取问题并产生答案;或者采取答案并产生问题;或者采取答案和问题,并产生多项选择选项。我们描述了系统,并说明了各种问题类型,它产生了令人惊讶的好答案,以及培训设置之外。我们还确定了问题类,它似乎仍然挣扎,提供洞察预训练的语言模型的局限性。Macaw是免费提供的,我们希望它对社区有用。Macaw可在https://github.com/allenai/macaw上获得

英文摘要:

Despite the successes of pretrained language models, there are still few high-quality, general-purpose QA systems that are freely available. In response, we present Macaw, a versatile, generative question-answering (QA) system that we are making available to the community. Macaw is built on UnifiedQA, itself built on T5, and exhibits strong performance, zero-shot, on a wide variety of topics, including outperforming GPT-3 by over 10% (absolute) on Challenge300, a suite of 300 challenge questions, despite being an order of magnitude smaller (11 billion vs. 175 billion parameters). In addition, Macaw allows different permutations ("angles") of its inputs and outputs to be used, for example Macaw can take a question and produce an answer; or take an answer and produce a question; or take an answer and question, and produce multiple-choice options. We describe the system, and illustrate a variety of question types where it produces surprisingly good answers, well outside the training setup. We also identify question classes where it still appears to struggle, offering insights into the limitations of pretrained language models. Macaw is freely available, and we hope that it proves useful to the community. Macaw is available at https://github.com/allenai/macaw

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑