arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.12271cs.DCcs.LG
收藏

跨异构云与高性能计算资源的安全联邦学习——以LLaMA 2联邦微调为例

Secure Federated Learning Across Heterogeneous Cloud and High-Performance Computing Resources -- A Case Study on Federated Fine-tuning of LLaMA 2

University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Argonne National Laboratory(阿贡国家实验室)

Zilinghan Li, Shilan He, Pranshu Chaturvedi, Volodymyr Kindratenko, Eliu A Huerta, Kibaek Kim, Ravi Madduri

AI总结:

本文提出APPFL框架,利用Globus Compute和AWS实现跨云与超算的安全联邦学习,并以微调LLaMA 2 7B模型为例验证其可行性。

摘要:

联邦学习使多个数据所有者能够协作训练鲁棒的机器学习模型,而无需传输大型或敏感的本地数据集,只需共享本地训练模型的参数。在本文中,我们详细阐述了我们的先进隐私保护联邦学习(APPFL)框架的设计,该框架通过利用Globus Compute(一个分布式函数即服务平台)和Amazon Web Services,简化了跨云计算设施和高性能计算资源的端到端安全可靠的联邦学习实验。我们进一步展示了APPFL在利用多个云资源和超级计算机微调LLaMA 2 7B模型中的应用案例。

英文摘要:

Federated learning enables multiple data owners to collaboratively train robust machine learning models without transferring large or sensitive local datasets by only sharing the parameters of the locally trained models. In this paper, we elaborate on the design of our Advanced Privacy-Preserving Federated Learning (APPFL) framework, which streamlines end-to-end secure and reliable federated learning experiments across cloud computing facilities and high-performance computing resources by leveraging Globus Compute, a distributed function as a service platform, and Amazon Web Services. We further demonstrate the use case of APPFL in fine-tuning a LLaMA 2 7B model using several cloud resources and supercomputers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑