联邦学习在生物医学自然语言处理中的深入评估
An In-Depth Evaluation of Federated Learning on Biomedical Natural Language Processing
AI总结：
本研究在2项生物医学NLP任务、8个语料库和6种语言模型上评估联邦学习，发现其优于单客户端训练模型，有时与集中训练相当，且比大型语言模型更快更优。
摘要：
语言模型（LMs）如BERT和GPT已彻底改变了自然语言处理（NLP）。然而，医学领域在训练语言模型时面临挑战，原因在于数据访问受限以及《健康保险携带与责任法案》（HIPPA）和《通用数据保护条例》（GDPR）等法规施加的隐私约束。联邦学习（FL）提供了一种去中心化的解决方案，能够在确保数据隐私的同时实现协作学习。在本研究中，我们在涵盖8个语料库的2项生物医学NLP任务上，使用6种语言模型评估了联邦学习。我们的结果表明：1）联邦学习模型始终优于在单个客户端数据上训练的模型，有时其性能与使用汇总数据训练的模型相当；2）在总数据量固定的情况下，使用更多客户端训练的联邦学习模型性能较差，但基于预训练Transformer的模型表现出很强的韧性；3）联邦学习模型显著优于使用零样本或单样本学习的大型语言模型，并提供了闪电般的推理速度。
英文摘要：
Language models (LMs) such as BERT and GPT have revolutionized natural language processing (NLP). However, the medical field faces challenges in training LMs due to limited data access and privacy constraints imposed by regulations like the Health Insurance Portability and Accountability Act (HIPPA) and the General Data Protection Regulation (GDPR). Federated learning (FL) offers a decentralized solution that enables collaborative learning while ensuring data privacy. In this study, we evaluated FL on 2 biomedical NLP tasks encompassing 8 corpora using 6 LMs. Our results show that: 1) FL models consistently outperformed models trained on individual clients' data and sometimes performed comparably with models trained with polled data; 2) with the fixed number of total data, FL models training with more clients produced inferior performance but pre-trained transformer-based models exhibited great resilience. 3) FL models significantly outperformed large language models using zero-/one-shot learning and offered lightning inference speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
