PEFT-MedAware:面向医学意识的大型语言模型
PEFT-MedAware: Large Language Model for Medical Awareness
AI总结:
本研究提出PEFT-MedAware模型,利用参数高效微调在MedQuAD数据上增强Falcon-1b,仅用0.44%可训练参数实现高精度医学问答,适合资源受限环境。
摘要:
聊天模型能够回答广泛的问题,但其回答的准确性高度不确定。在本研究中,我们提出了一种专门的PEFT-MedAware模型,利用参数高效微调(PEFT)在包含16,407个医学问答对的专用MedQuAD数据上增强Falcon-1b大型语言模型,仅使用其0.44%的可训练参数以提高计算效率。本文采用数据预处理和PEFT来优化模型性能,并辅以BitsAndBytesConfig以实现高效的Transformer训练。所得到的模型在特定领域的医学问答任务中,以更高的准确性超越了其他大型语言模型,同时利用有限的计算资源,使其适合部署在资源受限的环境中。我们提出通过扩展数据集、更大的模型和反馈机制来持续保持医学相关性。我们的工作凸显了PEFT在医学AI中的效率提升和专门能力,在精度上超越标准模型而无需大量资源需求。所提出的模型和数据仅用于研究目的发布。
英文摘要:
Chat models are capable of answering a wide range of questions, however, the accuracy of their responses is highly uncertain. In this research, we propose a specialized PEFT-MedAware model where we utilize parameter-efficient fine-tuning (PEFT) to enhance the Falcon-1b large language model on specialized MedQuAD data consisting of 16,407 medical QA pairs, leveraging only 0.44% of its trainable parameters to enhance computational efficiency. The paper adopts data preprocessing and PEFT to optimize model performance, complemented by a BitsAndBytesConfig for efficient transformer training. The resulting model was capable of outperforming other LLMs in medical question-answering tasks in specific domains with greater accuracy utilizing limited computational resources making it suitable for deployment in resource-constrained environments. We propose further improvements through expanded datasets, larger models, and feedback mechanisms for sustained medical relevancy. Our work highlights the efficiency gains and specialized capabilities of PEFT in medical AI, outpacing standard models in precision without extensive resource demands. The proposed model and data are released for research purposes only.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。