Multitask Prompted Training Enables Zero-Shot Task Generalization
Hugging Face;Brown University(布朗大学);Snorkel AI;BigScience;KFUPM(法赫德国王石油与矿业大学);IRISA(法国信息与随机系统研究所);IMATAG;Hyperscience;SAP(思爱普);NTU(南洋理工大学);UCSD(加州大学圣地亚哥分校);SambaNova Systems;Walmart Labs(沃尔玛实验室);VU Amsterdam(阿姆斯特丹自由大学);University of Virginia(弗吉尼亚大学);ASUS(华硕);ZEALS;NYU(纽约大学);IBM Research(IBM研究院);UC Berkeley(加州大学伯克利分校);Parity;Inria(法国国家信息与自动化研究所);Naver Labs Europe(Naver欧洲实验室);BITS Pilani(博拉理工学院);University of Rome(罗马大学);Stanford(斯坦福大学);Charles River Analytics(查尔斯河分析公司);Booz Allen(博思艾伦);EleutherAI
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan De… 展开作者
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Tali Bers, Stella Biderman, Leo Gao, Thomas Wolf, Alexander M. Rush
Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is a consequence of implicit multitask learning in language models' pretraining (Radford et al., 2019). Can zero-shot generalization instead be directly induced by explicit multitask learning? To test this question at scale, we develop a system for easily mapping any natural language tasks into a human-readable prompted form. We convert a large set of supervised datasets, each with multiple prompts with diverse wording. These prompted datasets allow for benchmarking the ability of a model to perform completely held-out tasks. We fine-tune a pretrained encoder-decoder model (Raffel et al., 2020; Lester et al., 2021) on this multitask mixture covering a wide variety of tasks. The model attains strong zero-shot performance on several standard datasets, often outperforming models up to 16x its size. Further, our approach attains strong performance on a subset of tasks from the BIG-bench benchmark, outperforming models up to 6x its size. All trained models are available at https://github.com/bigscience-workshop/t-zero and all prompts are available at https://github.com/bigscience-workshop/promptsource.
TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish
Technical University of Munich(慕尼黑工业大学);Center for Information and Language Processing, LMU Munich(慕尼黑大学信息与语言处理中心);Munich Center for Machine Learning(慕尼黑机器学习中心);Language Technology Lab, University of Cambridge(剑桥大学语言技术实验室)
xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval
Islamic University of Technology (IUT)(伊斯兰科技大学);Nanyang Technological University (NTU)(南洋理工大学);Qatar Computing Research Institute (QCRI)(卡塔尔计算研究所);Bosch Research(博世研究院);Salesforce Research(赛富时研究院)
Bidirectional Language Models Are Also Few-shot Learners
University of Pennsylvania(宾夕法尼亚大学);Microsoft(微软);Google Research(谷歌研究院);UNC Chapel Hill(北卡罗来纳大学教堂山分校)
大语言模型,例如GPT-3(Brown等人,2020)可以执行任意任务,而无需在仅使用几个标记示例进行提示后进行微调。一个任意的任务可以被重新表述为一个自然语言提示,一个语言模型可以被要求生成完成,间接地执行任务的范式称为基于语义的学习。到目前为止,基于突现的学习能力主要是针对单向语言模型来证明的。然而,在去噪目标上预训练的双向语言模型(例如掩蔽语言建模)为迁移学习产生更强的学习表示。这激发了促进双向模式的可能性,但它们的预培训目标使它们在很大程度上与现有的激励范式不相容。我们提出了SAP(顺序自回归拟合),一种技术,使双向模型的提示。利用机器翻译任务作为案例研究,我们提出了双向mT5模型(Xue et al.,2021)与SAP进行比较,并证明其Few-Shot和zero-shot转换优于单向模型(如GPT-3和XGLM)的Few-Shot转换(Lin et al.,2021),尽管mT5的参数减少了约50%。我们进一步表明SAP在问答和摘要方面是有效的。这是第一次,我们的研究结果表明,基于语义的学习是一个更广泛的一类语言模型,而不仅仅是单向模型的新兴属性。
ClaimIQ at CheckThat! 2025: Comparing Prompted and Fine-Tuned Language Models for Verifying Numerical Claims
Department of Data Science, University of North Texas, Denton, TX, USA(数据科学系,北卡罗来纳州立大学,德顿,得克萨斯州,美国);Department of Computer Science and Engineering, University of North Texas, Denton, TX, USA(计算机科学与工程系,北卡罗来纳州立大学,德顿,得克萨斯州,美国);Department of Computer Science(计算机科学系)
Unleashing Large Language Models' Proficiency in Zero-shot Essay Scoring
National Key Laboratory for Multimedia Information Processing, Peking University(北京大学多媒体信息处理国家重点实验室);School of Computer Science, Peking University(北京大学计算机学院);School of Software and Microelectronics, Peking University(北京大学软件与微电子学院)
In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization
Nanyang Technological University(南洋理工大学);Agency for Science, Technology and Research(新加坡科技研究局);Singapore Institute of Manufacturing Technology (SIMTech)(新加坡制造技术研究院);Centre for Frontier AI Research, A*STAR(A*STAR前沿人工智能研究中心)