arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.03103cs.CLcs.AI
收藏

语言模型的零镜头功能组合性

Towards Zero-Shot Functional Compositionality of Language Models

Riiid AI Research(Riiid人工智能研究院);NAVER AI Lab(NAVER人工智能实验室)

Hangyeol Yu, Myeongho Jeong, Jamin Shin, Hyeongdon Moon, Juneyoung Park, Seungtaek Choi

摘要:

大型预训练语言模型(PLM)已经成为NLP领域最理想的起点,因为它们在解决许多单独的任务方面变得非常出色。尽管取得了这样的成功,在本文中,我们认为,目前的模式与PLM的工作忽视了一个关键方面的建模人类智能:功能组合。功能组合性(functional compositionality)--组合学习任务的能力--一直是人工智能领域(以及许多其他领域)的一个长期挑战,因为它被认为是人类智能的标志之一。这样的说明性示例是跨语言摘要,其中双语人(英语-法语)可以直接将英语文档摘要成法语句子,而不必将英语文档或摘要明确地翻译成法语。我们讨论为什么这件事是一个重要的开放的问题,需要进一步关注的领域。然后,我们表明,目前的PLM(例如,GPT-2和T5)还没有功能组合性,离人类水平的概括性还很远。最后,我们提出了几个研究方向,可以推动该领域的zero-shot功能组合的语言模型。

英文摘要:

Large Pre-trained Language Models (PLM) have become the most desirable starting point in the field of NLP, as they have become remarkably good at solving many individual tasks. Despite such success, in this paper, we argue that current paradigms of working with PLMs are neglecting a critical aspect of modeling human intelligence: functional compositionality. Functional compositionality - the ability to compose learned tasks - has been a long-standing challenge in the field of AI (and many other fields) as it is considered one of the hallmarks of human intelligence. An illustrative example of such is cross-lingual summarization, where a bilingual person (English-French) could directly summarize an English document into French sentences without having to translate the English document or summary into French explicitly. We discuss why this matter is an important open problem that requires further attention from the field. Then, we show that current PLMs (e.g., GPT-2 and T5) don't have functional compositionality yet and it is far from human-level generalizability. Finally, we suggest several research directions that could push the field towards zero-shot functional compositionality of language models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑