Deep neural language models have set new breakthroughs in many tasks of Natural Language Processing (NLP). Recent work has shown that deep transformer language models (pretrained on large amounts of texts) can achieve high levels of task-specific few-shot performance comparable to state-of-the-art models. However, the ability of these large language models in few-shot transfer learning has not yet been explored in the biomedical domain. We investigated the performance of two powerful transformer language models, i.e. GPT-3 and BioBERT, in few-shot settings on various biomedical NLP tasks. The experimental results showed that, to a great extent, both the models underperform a language model fine-tuned on the full training data. Although GPT-3 had already achieved near state-of-the-art results in few-shot knowledge transfer on open-domain NLP tasks, it could not perform as effectively as BioBERT, which is orders of magnitude smaller than GPT-3. Regarding that BioBERT was already pretrained on large biomedical text corpora, our study suggests that language models may largely benefit from in-domain pretraining in task-specific few-shot learning. However, in-domain pretraining seems not to be sufficient; novel pretraining and few-shot learning strategies are required in the biomedical NLP domain.
Holistic Evaluation of GPT-4V for Biomedical Imaging
University of Georgia（佐治亚大学）；Beijing Jiaotong University（北京交通大学）；Northwestern Polytechnical University（西北工业大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Electronic Science and Technology of China（电子科技大学）
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
LMU Munich（慕尼黑大学）；Sulzer GmbH（苏尔寿有限公司）
当缩放到数千亿个参数时，预训练的语言模型，例如GPT-3（Brown et al.，2020）实现了卓越的Few-Shot性能。然而，训练和应用这些大型模型需要大量的计算，导致大量的碳足迹，使研究人员和从业人员难以使用它们。我们表明，可以获得类似于GPT-3的性能与语言模型，是“绿色”，因为他们的参数计数是几个数量级小。这是通过将文本输入转换为包含任务描述的完形填空题，结合基于梯度的优化来实现的;利用未标记的数据可以进一步改进。我们确定成功的自然语言理解所需的关键因素与小语言模型。
Large Language Model-Driven Dynamic Assessment of Grammatical Accuracy in English Language Learner Writing
National Institute of Standards（国家标准 institute）；Department of Physics, Colorado State University（物理系，科罗拉多州立大学）；Electrical Engineering Department, University of Colorado（电气工程系，科罗拉多大学）
Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution
Division of Intramural Research, National Library of Medicine, National Institutes of Health（国家医学图书馆内部研究部，国立卫生研究院）；Department of Computer Science, University of Virginia（弗吉尼亚大学计算机科学系）；Center for Cancer Research, National Cancer Institute, National Institutes of Health（国家癌症研究所癌症研究中心，国立卫生研究院）；Department of Population Health Sciences, Weill Cornell Medicine Institute of AI for Digital Health, Weill Cornell Medicine（韦尔·科恩医学中心流行病学与健康科学系，韦尔·科恩医学中心人工智能与数字健康研究所）
UM_FHS at TREC 2024 PLABA: Exploration of Fine-tuning and AI agent approach for plain language adaptations of biomedical text
University of Maribor（马里博尔大学）；University of Ljubljana（卢布尔雅那大学）；Columbia University（哥伦比亚大学）；Manisa Celal Bayar University（马尼萨杰拉尔·巴亚尔大学）；University of Edinburgh（爱丁堡大学）
Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification
Mass General Brigham, Harvard Medical School（麻省总医院布莱根，哈佛医学院）；Brigham and Women’s Hospital/Dana-Farber Cancer Institute（布莱根妇女医院/丹娜-法伯癌症研究所）；Technische Universität Darmstadt（达姆施塔特工业大学）；Maastricht University（马斯特里赫特大学）
Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large language models
Department of Biomedical Informatics, School of Medicine, Emory University（生物医学信息学系，医学院，埃默里大学）；Department of Computer Science, Emory University（计算机科学系，埃默里大学）；National Library of Medicine, National Institutes of Health（国家医学图书馆，国立卫生研究院）；Department of Biomedical Engineering, Georgia Institute of Technology and Emory University（生物医学工程系，佐治亚理工学院和埃默里大学）
Benchmarking large language models for biomedical natural language processing applications and recommendations
Yale University（耶鲁大学）；National Institutes of Health（美国国立卫生研究院）；University of Texas Health Science at Houston（休斯顿德克萨斯大学健康科学中心）；University of Minnesota（明尼苏达大学）
Tabular LLMs for Interpretable Few-Shot Alzheimer's Disease Prediction with Multimodal Biomedical Data
University of Pennsylvania（宾夕法尼亚大学）；Stony Brook University（石溪大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Medical University of South Carolina（南卡罗来纳医科大学）；Cedars-Sinai Medical Center（西格拉姆山医疗中心）
Biomedical knowledge graph-optimized prompt generation for large language models
University of California San Francisco（加州大学旧金山分校）；San Diego Supercomputer Center, University of California, San Diego（加州大学圣地亚哥分校圣地亚哥超级计算机中心）；University of California, San Francisco（加州大学旧金山分校）；Institute for Systems Biology（系统生物学研究所）；Bakar Computational Health Sciences Institute, University of California, San Francisco（加州大学旧金山分校巴卡尔计算健康科学研究所）；Mate Bioservices, Inc.（Mate 生物服务公司）
Cancer Vaccine Adjuvant Name Recognition from Biomedical Literature using Large Language Models
University of North Dakota School of Medicine and Health Sciences（北达科他大学医学与健康科学学院）；University of North Dakota（北达科他大学）；University of Michigan（密歇根大学）；Bogazici University（博阿齐奇大学）
Mapping Biomedical Ontology Terms to IDs: Effect of Domain Prevalence on Prediction Accuracy
Computer Science Department（计算机科学系）；Missouri State University（密苏里州立大学）；Kummer Institute Center for AI and AS（Kummer研究所人工智能与自动系统中心）；Missouri University of Science & Technology（密苏里科技大学）；Cooperative Engineering Program（合作工程计划）
MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Science
Emory University（埃默里大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Yale University（耶鲁大学）；UT Southwestern Medical Center（德克萨斯大学西南医学中心）；University of Washington（华盛顿大学）
From Knowledge Generation to Knowledge Verification: Examining the BioMedical Generative Capabilities of ChatGPT
MGEN -- College of Engineering, Northeastern University Miami（东北大学迈阿密分校MGEN工程学院）；University of Vermont（佛蒙特大学）；AGH, University of Krakow（克拉科夫AGH大学）；Howard University（霍华德大学）
GP-GPT: Large Language Model for Gene-Phenotype Mapping
Department of Computer Science and Engineering, The University of Texas at Arlington（计算机科学与工程系，德克萨斯理工大学 Arlington 分校）；School of Computing, University of Georgia（计算学院，佐治亚大学）；Department of Computer Science, Indiana University Indianapolis（计算机科学系，印第安纳大学 Indianapolis 分校）；Department of Radiology, Massachusetts General Hospital and Harvard Medical School（放射科，麻省总医院和哈佛医学院）；Department of Statistics, University of Georgia（统计系，佐治亚大学）；Department of Epidemiology & Biostatistics, University of Georgia（流行病学与生物统计学系，佐治亚大学）；Department of Environmental Health Science, University of Georgia（环境流行病学系，佐治亚大学）
Benchmarking and Analyzing In-context Learning, Fine-tuning and Supervised Learning for Biomedical Knowledge Curation: a focused study on chemical entities of biological interest
Institute of Health Informatics（健康信息学研究所）；University College London（伦敦大学学院）
Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems
Center for Artificial Intelligence Research (CAiRE)（人工智能研究中心）；The Hong Kong University of Science and Technology（香港科技大学）
面向任务的对话系统使用四个连接的模块，即，自然语言理解（NLU）、对话状态跟踪（DST）、对话策略（DP）和自然语言生成（NLG）。研究的挑战是用最少的样本学习每个模块（即，考虑到与数据汇集相关的高成本，解决这个问题最常见和最有效的技术是迁移学习，其中大语言模型，无论是在文本或特定于任务的数据上进行预训练，都是在少数样本上进行微调。这些方法需要对每个任务进行微调步骤和一组参数。因此，语言模型，例如GPT-2（Radford et al.，2019）和GPT-3（Brown等人，2020），允许通过用很少的例子启动模型来进行少样本学习。在本文中，我们评估了语言模型在NLU，DST，DP和NLG任务中的启动Few-Shot能力。重要的是，我们强调了这种方法目前的局限性，我们讨论了未来工作的可能影响。
Bidirectional Language Models Are Also Few-shot Learners
University of Pennsylvania（宾夕法尼亚大学）；Microsoft（微软）；Google Research（谷歌研究院）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）
大语言模型，例如GPT-3（Brown等人，2020）可以执行任意任务，而无需在仅使用几个标记示例进行提示后进行微调。一个任意的任务可以被重新表述为一个自然语言提示，一个语言模型可以被要求生成完成，间接地执行任务的范式称为基于语义的学习。到目前为止，基于突现的学习能力主要是针对单向语言模型来证明的。然而，在去噪目标上预训练的双向语言模型（例如掩蔽语言建模）为迁移学习产生更强的学习表示。这激发了促进双向模式的可能性，但它们的预培训目标使它们在很大程度上与现有的激励范式不相容。我们提出了SAP（顺序自回归拟合），一种技术，使双向模型的提示。利用机器翻译任务作为案例研究，我们提出了双向mT5模型（Xue et al.，2021）与SAP进行比较，并证明其Few-Shot和zero-shot转换优于单向模型（如GPT-3和XGLM）的Few-Shot转换（Lin et al.，2021），尽管mT5的参数减少了约50%。我们进一步表明SAP在问答和摘要方面是有效的。这是第一次，我们的研究结果表明，基于语义的学习是一个更广泛的一类语言模型，而不仅仅是单向模型的新兴属性。
Biomedical Visual Instruction Tuning with Clinician Preference Alignment
Stanford University（斯坦福大学）；Emory University（埃默里大学）；University of California, Berkeley（加州大学伯克利分校）；University of Washington（华盛顿大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Enabling Few-Shot Alzheimer's Disease Diagnosis on Biomarker Data with Tabular LLMs
University of Pennsylvania（宾夕法尼亚大学）；United States Naval Academy（美国海军学院）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Cedars Sinai Medical Center（西好莱坞 Cedars Sinai 医院）
Knowledge-Driven Agentic Scientific Corpus Distillation Framework for Biomedical Large Language Models Training
Computer Network Information Center, Chinese Academy of Sciences, Beijing, China（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学）；Duke-NUS Medical School, National University of Singapore, Singapore（新加坡国立大学杜克-新加坡医学部）
Predicting Failures of LLMs to Link Biomedical Ontology Terms to Identifiers Evidence Across Models and Ontologies
Department of Neurology（神经学系）；Rehabilitation University of Illinois at Chicago Chicago, IL, USA（伊利诺伊大学芝加哥分校康复学院）；Lab for Applied Artificial Intelligence（应用人工智能实验室）；Engineering Program（工程学院）
