We view the landscape of large language models (LLMs) through the lens of the recently released BLOOM model to understand the performance of BLOOM and other decoder-only LLMs compared to BERT-style encoder-only models. We achieve this by evaluating the smaller BLOOM model variants (\textit{350m/560m} and \textit{1b3/1b7}) on several NLP benchmark datasets and popular leaderboards. We make the following observations: (1) BLOOM performance does not scale with parameter size, unlike other LLMs like GPT and BERT. Experiments fine-tuning BLOOM models show that the 560m variant performs similarly to or better than the 1b7 variant, (2) Zero-shot cross-lingual and multi-lingual fine-tuning experiments show that BLOOM is at par or worse than monolingual GPT-2 models, and (3) Toxicity analysis of prompt-based text generation using the RealToxicityPrompts dataset shows that the text generated by BLOOM is at least 17\% less toxic than GPT-2 and GPT-3 models.
Benchmarking the Performance of Pre-trained LLMs across Urdu NLP Tasks
Center for Language Engineering（语言工程中心）；Al-Khawarizmi Institute of Computer Science（阿尔-花剌子密计算机科学研究所）；University of Engineering and Technology Lahore（拉合尔工程技术大学）
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Pattern Recognition Center, WeChat AI, Tencent Inc（腾讯公司微信AI模式识别中心）；School of Engineering, Westlake University（西湖大学工程学院）；Institute of Advanced Technology, Westlake Institute for Advanced Study（西湖高等研究院先进技术研究所）
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
University of Amsterdam（阿姆斯特丹大学）；University of Trento（特伦托大学）；University of Copenhagen（哥本哈根大学）；Utrecht University（乌得勒支大学）；Max Planck Institute for Psycholinguistics（马克斯·普朗克心理语言学研究所）；ETH Zürich（苏黎世联邦理工学院）；Saarland University（萨尔大学）；Universidade de Lisboa & Unbabel（里斯本大学及Unbabel）；LMU Munich & MCML（慕尼黑大学及MCML）；University of Potsdam（波茨坦大学）；Heriot-Watt University（赫瑞线学院）
MoZIP: A Multilingual Benchmark to Evaluate Large Language Models in Intellectual Property
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen Technology University（深圳技术大学）；Shenzhen University（深圳大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
Do Prompts Solve NLP Tasks Using Natural Language?
Westlake University（西湖大学）；University of Electronic Science and Technology of China（电子科技大学）；Zhejiang University（浙江大学）；Westlake Institute for Advanced Study（西湖高等研究院）
TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise
University of the Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；University of Electronic Science and Technology of China（电子科技大学）；The University of Hong Kong（香港大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；SenseTime Research（商汤科技研究院）
An Empirical Study on Information Extraction using Large Language Models
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University（吉林大学符号计算与知识工程教育部重点实验室）；Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）大数据研究院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）数据科学学院）；College of Software, Jilin University（吉林大学软件学院）；School of Information Technology, Halmstad University, Sweden（哈尔姆斯塔德大学信息技术学院）
Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study
Technical University of Munich, School of Computation, Information and Technology, Department of Computer Science, Munich, Germany（慕尼黑技术大学，计算、信息与技术学院，计算机科学系，德国慕尼黑）
NLP Verification: Towards a General Methodology for Certifying Robustness
Heriot-Watt University（赫瑞瓦特大学）；University of Southampton（南安普顿大学）；University of Birmingham（伯明翰大学）；University of Western Australia（西澳大学）；The Hebrew University of Jerusalem（耶路撒冷希伯来大学）
Empirical Evaluation of Multi-task Learning in Deep Neural Networks for Natural Language Processing
Beijing Ultrapower Software Co.,Ltd.（北京神州泰岳软件股份有限公司）；University of Pennsylvania（宾夕法尼亚大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
An Empirical Comparison of Instance Attribution Methods for NLP
University of California, Irvine（加州大学尔湾分校）；Northeastern University（东北大学）
深度模型的广泛采用激发了对解释网络输出和促进模型调试的方法的迫切需求。实例归因方法通过检索（可能）导致特定预测的训练实例来实现这些目标。影响函数（IF; Koh and Liang 2017）通过量化扰动单个训练实例对特定测试预测的影响，提供了实现这一点的机制。然而，即使近似IF也是计算上昂贵的，在许多情况下可能是禁止的。更简单的方法（例如，检索与给定测试点最相似的训练示例）执行测试？在这项工作中，我们评估的程度不同的潜在实例属性同意训练样本的重要性。我们发现，简单的检索方法产生的训练实例与通过基于梯度的方法（如IF）识别的训练实例不同，但仍然表现出与更复杂的归因方法相似的理想特征。本文中所有方法和实验的代码可在https://github.com/successar/instance_attributions_NLP上获得。
Perspectives on Cross-Lingual Consistency in LLMs for Medical Questions
Johannes Gutenberg University Mainz（美因茨约翰内斯·古腾堡大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；University of Marburg（马尔堡大学）；Universidad Iberoamericana（伊比利亚美洲大学）
