arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2506.20009cs.AIcs.CL
收藏

准确且节能:本地检索增强生成模型在医疗任务中优于商业大语言模型

Accurate and Energy Efficient: Local Retrieval-Augmented Generation Models Outperform Commercial Large Language Models in Medical Tasks

University of Crete(克里特大学);University Hospital of Heraklion(伊拉克利翁大学医院);Karolinska Institute(卡罗林斯卡学院)

Konstantinos Vrettos, Michail E. Klontzas

AI总结:

本研究提出一种可监控能耗的模块化医疗检索增强生成(RAG)框架,基于本地开源大模型构建的RAG在医疗问答任务中的准确率和能效均优于商业大模型,推动了可持续AI发展。

摘要:

背景:人工智能(AI)在医疗保健领域的日益普及引发了人们对其环境与伦理影响的日益担忧。商业大语言模型(LLMs),如ChatGPT和DeepSeek,需要大量资源,而将这些系统用于医疗目的引发了关于患者隐私和安全的关键问题。方法:我们为医疗任务开发了一个可定制的检索增强生成(RAG)框架,该框架可监控其能源使用和CO2排放。随后使用该系统基于各种开源LLM创建了RAG。测试的模型包括通用模型(如llama3.1:8b)和医疗领域专用模型medgemma-4b-it。将最佳RAG的性能和能耗与DeepSeekV3-R1和OpenAI的o4-mini模型进行了比较。评估使用了一个医疗问题数据集。结果:定制RAG模型在准确性和能耗方面均优于商业模型。基于llama3.1:8B构建的RAG模型实现了最高准确率(58.5%),且明显优于包括o4-mini和DeepSeekV3-R1在内的其他模型。llama3.1-RAG在所有模型中还表现出最低的能耗和CO2足迹,其每千瓦时性能为0.52,总CO2排放量为473g。与o4-mini相比,llama3.1-RAG在保持更高准确率的同时,每千瓦时获得的准确率点数提高了2.7倍,且用电量减少了172%。结论:我们的研究表明,可以利用本地LLM开发RAG,使其在医疗任务中优于商业在线LLM,同时具有较小的环境影响。我们的模块化框架促进了可持续AI的发展,减少了用电量,并符合联合国的可持续发展目标。

英文摘要:

Background The increasing adoption of Artificial Intelligence (AI) in healthcare has sparked growing concerns about its environmental and ethical implications. Commercial Large Language Models (LLMs), such as ChatGPT and DeepSeek, require substantial resources, while the utilization of these systems for medical purposes raises critical issues regarding patient privacy and safety. Methods We developed a customizable Retrieval-Augmented Generation (RAG) framework for medical tasks, which monitors its energy usage and CO2 emissions. This system was then used to create RAGs based on various open-source LLMs. The tested models included both general purpose models like llama3.1:8b and medgemma-4b-it, which is medical-domain specific. The best RAGs performance and energy consumption was compared to DeepSeekV3-R1 and OpenAIs o4-mini model. A dataset of medical questions was used for the evaluation. Results Custom RAG models outperformed commercial models in accuracy and energy consumption. The RAG model built on llama3.1:8B achieved the highest accuracy (58.5%) and was significantly better than other models, including o4-mini and DeepSeekV3-R1. The llama3.1-RAG also exhibited the lowest energy consumption and CO2 footprint among all models, with a Performance per kWh of 0.52 and a total CO2 emission of 473g. Compared to o4-mini, the llama3.1-RAG achieved 2.7x times more accuracy points per kWh and 172% less electricity usage while maintaining higher accuracy. Conclusion Our study demonstrates that local LLMs can be leveraged to develop RAGs that outperform commercial, online LLMs in medical tasks, while having a smaller environmental impact. Our modular framework promotes sustainable AI development, reducing electricity usage and aligning with the UNs Sustainable Development Goals.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑