Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 recent studies of English language model behavior before task-specific fine-tuning. Language models possess basic capabilities in syntax, semantics, pragmatics, world knowledge, and reasoning, but these capabilities are sensitive to specific inputs and surface features. Despite dramatic increases in generated text quality as models scale to hundreds of billions of parameters, the models are still prone to unfactual responses, commonsense errors, memorized text, and social biases. Many of these weaknesses can be framed as over-generalizations or under-generalizations of learned patterns in text. We synthesize recent results to highlight what is currently known about large language model capabilities, thus providing a resource for applied work and for research in adjacent fields that use language models.
A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery
University of Washington(华盛顿大学);Texas A&M University(德克萨斯农工大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)
A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models
Amirkabir University of Technology(阿米尔卡比尔理工大学);Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院);Columbia University(哥伦比亚大学);Amazon GenAI(亚马逊生成式AI部门)
A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(莫扎德大学人工智能学院);Fraunhofer Institute for Open Communication Systems (FOKUS)(弗劳恩霍夫开放通信系统研究所);Technical University of Munich(慕尼黑技术大学);Technical University of Darmstadt(达姆施塔特技术大学);University of Toronto(多伦多大学)
Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);McGill University(麦吉尔大学);Tencent Company(腾讯公司);Artificial Intelligence Research Institute, iFLYTEK Co., Ltd(人工智能研究院,iFLYTEK公司)
A Comprehensive Survey on Aspect Based Sentiment Analysis
Vellore Institute Of Technology(韦洛尔理工大学);School of Computer Science and Engineering- SCOPE(计算机科学与工程学院-SCOPE)
基于方面的情感分析(Aspect Based Sentiment Analysis,ABSA)是自然语言处理的一个分支,它主要是将数据分成方面,并最终提取情感信息。已知ABSA比一般的情感分析提供更多关于上下文的信息。在这项研究中,我们的目的是探索各种方法,同时执行ABSA,并提供一个比较研究。这份调查报告深入讨论了各种解决方案,并对它们进行了比较。并方便地分为几个部分,以获得对该过程的整体看法。