MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；Aalto University（阿尔托大学）；The University of Melbourne（墨尔本大学）；Linköping University（林雪平大学）
Omkar Thawakar, Ashmal Vayani, Salman Khan, Hisham Cholakal, Rao M. Anwer, Michael Felsberg, Tim Baldwin, Eric P. Xing, Fahad Shahbaz Khan
"Bigger the better" has been the predominant trend in recent Large Language Models (LLMs) development. However, LLMs do not suit well for scenarios that require on-device processing, energy efficiency, low memory footprint, and response efficiency. These requisites are crucial for privacy, security, and sustainable deployment. This paper explores the "less is more" paradigm by addressing the challenge of designing accurate yet efficient Small Language Models (SLMs) for resource constrained devices. Our primary contribution is the introduction of an accurate and fully transparent open-source 0.5 billion (0.5B) parameter SLM, named MobiLlama, catering to the specific needs of resource-constrained computing with an emphasis on enhanced performance with reduced resource demands. MobiLlama is a SLM design that initiates from a larger model and applies a careful parameter sharing scheme to reduce both the pre-training and the deployment cost. Our work strives to not only bridge the gap in open-source SLMs but also ensures full transparency, where complete training data pipeline, training code, model weights, and over 300 checkpoints along with evaluation codes is available at : https://github.com/mbzuai-oryx/MobiLlama.
Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
NYU Langone Health（纽约大学朗格尼医学中心）；The University of Texas at Austin（得克萨斯大学奥斯汀分校）；Washington University School of Medicine in St. Louis（圣路易斯华盛顿大学医学院）；New York University（纽约大学）
LePREC: Reasoning as Classification over Structured Factors for Assessing Relevance of Legal Issues
Faculty of Information Technology, Monash University, Australia（墨尔本大学信息技术学院，澳大利亚）；School of Information Technology, Monash University Malaysia（墨尔本大学马来西亚分校信息学院）；School of Business, Monash University Malaysia（墨尔本大学马来西亚分校商学院）；Faculty of Law, Monash University, Australia（墨尔本大学法学院，澳大利亚）
GOFAI meets Generative AI: Development of Expert Systems by means of Large Language Models
Quantitative Methods Department, Comillas Pontifical University Institute of Research in Technology (IIT)（定量方法部门，科利马斯天主教大学技术研究所）；Computer Science Department, ICAI School of Engineering, Comillas Pontifical University（计算机科学系，ICAI工程学院，科利马斯天主教大学）
A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；Case Western Reserve University（凯斯西储大学）；University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）；Stony Brook University（纽约州立大学石溪分校）；University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Carnegie Mellon University（卡内基梅隆大学）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Northeastern University（东北大学）；University of California, Berkeley（加利福尼亚大学伯克利分校）；ServiceNow Research（ServiceNow研究院）；Hugging Face；Roblox；Cursor AI
Multi-LLM Thematic Analysis with Dual Reliability Metrics: Combining Cohen's Kappa and Semantic Similarity for Qualitative Research Validation
AZA Lab, Department of Psychiatry, Yale University（AZA实验室、精神病学系、耶鲁大学）；Department of Psychiatry and Behavioral Sciences, Center for Collective Healing, Howard University（精神病学与行为科学系、集体愈合中心、霍华德大学）；Department of Psychology, University of Exeter（心理学系、埃克塞特大学）
Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding
Vector Institute for Artificial Intelligence（矢量人工智能研究所）；University of Toronto（多伦多大学）；Peter Munk Cardiac Centre, University Health Network（大学健康网络彼得·蒙克心脏中心）；McGill University（麦吉尔大学）；University Health Network（大学健康网络）
Evaluating the Ability of Large Language Models to Identify Adherence to CONSORT Reporting Guidelines in Randomized Controlled Trials: A Methodological Evaluation Study
SUN YAT-SEN MEMORIAL HOSPITAL（中山纪念医院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Imperial College London（伦敦帝国学院）
FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets
University of California, Los Angeles（加州大学洛杉矶分校）；NYU Shanghai（上海纽约大学）；Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning（上海人工智能与深度学习前沿科学中心）；Columbia University（哥伦比亚大学）
DeepSeek in Healthcare: A Survey of Capabilities, Risks, and Clinical Applications of Open-Source Large Language Models
Weill Cornell Medicine, Cornell University（威尔康奈尔医学院，康奈尔大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Touro University College of Osteopathic Medicine（图罗大学整骨医学院）
Seeds of Stereotypes: A Large-Scale Textual Analysis of Race and Gender Associations with Diseases in Online Sources
Aarhus University（奥胡斯大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Alberta（阿尔伯塔大学）；University of Manitoba（曼尼托巴大学）；Mbarara University of Science and Technology（姆巴拉拉科技大学）；Fundacion Arturo Lopez Perez（阿图罗·洛佩斯·佩雷斯基金会）；Mass General Brigham（麻省总医院布里格姆）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Emory University School of Medicine（埃默里大学医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）
本文通过分析 Arxiv、Wikipedia 和 Common Crawl 中18种疾病与种族、性别术语的共现，并对比患病率与 GPT-4 输出，揭示在线文本及 LLM 预训练数据中的人口统计学偏见。
Breaking the Chains of Probability: Neutrosophic Logic as a New Framework for Epistemic Uncertainty in Large Language Models
Universidad Bolivariana del Ecuador, Coordinación Académica de Posgrado（巴尔干大学厄瓜多尔分校，研究生院）；Universidad de Guayaquil（瓜亚基尔大学）；Universidad Bernardo O’Higgins（伯纳多·奥希金斯大学）；Mathematics, Physics, and Natural Sciences Division, University of New Mexico（新墨西哥大学数学、物理和自然科学系）
