arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.04243cs.DC
收藏

FedOptimus:优化垂直联邦学习的可扩展性与效率

FedOptimus: Optimizing Vertical Federated Learning for Scalability and Efficiency

Nikita Shrivastava, Drishya Uniyal, Bapi Chatterjee

AI总结:

针对垂直联邦学习(VFL)的通信瓶颈、计算低效和非IID数据收敛慢等问题,本文提出FedOptimus多VFL框架,通过互信息客户端选择、服务端动量技术及K步平均优化,在基准数据集上提升了可扩展性与效率。

摘要:

联邦学习(FL)是一种协作式机器学习范式,通过在分布式数据集上训练模型而不集中敏感信息来确保数据隐私。垂直联邦学习(VFL)是FL的一种训练方法,促进参与者之间的协作,每个客户端拥有共享用户集的不同特征空间。因此,VFL在金融和医疗等隐私敏感领域具有不可估量的价值。尽管有固有优势,VFL仍面临通信瓶颈、计算效率低下以及非IID数据分布导致的收敛缓慢等挑战。本文介绍FedOptimus,一个稳健的多VFL框架,集成先进技术以提升模型效率和可扩展性。FedOptimus利用基于互信息(MI)的客户端选择来优先考虑高贡献参与者,减少计算开销;进一步纳入FedAvgM和SLOWMO等服务端动量技术,在异构数据上稳定更新并加速收敛;此外,执行K步平均在维持模型性能的同时最小化通信成本。FedOptimus在CIFAR-10、MNIST和FMNIST等基准数据集上表现更优,展示了其在真实多服务器、多客户端场景中的可扩展性和有效性。通过统一先进优化方法,FedOptimus为高效且可扩展的垂直联邦学习框架树立新标准,为其在复杂隐私敏感领域的更广泛应用铺路。

英文摘要:

Federated learning (FL) is a collaborative machine learning paradigm which ensures data privacy by training models across distributed datasets without centralizing sensitive information. Vertical Federated Learning (VFL), a kind of FL training method, facilitates collaboration among participants with each client having received a different feature space of a shared user set. VFL thus, proves invaluable in privacy-sensitive domains such as finance and healthcare. Despite its inherent advantages, VFL faced challenges including communication bottlenecks, computational inefficiency, and slow convergence due to non-IID data distributions. This paper introduces FedOptimus, a robust Multi-VFL framework integrating advanced techniques for improved model efficiency and scalability. FedOptimus leverages a Mutual Information (MI)-based client selection to prioritize high-contribution participants, reducing computational overhead. Further, it incorporates server-side momentum techniques like FedAvgM and SLOWMO to stabilize updates and accelerate convergence on heterogeneous data. Additionally, performing K-Step Averaging minimizes communication costs while maintaining model performance. FedOptimus proves to be superior in performance on benchmark datasets such as CIFAR-10, MNIST, and FMNIST, showcasing its scalability and effectiveness in real-world multi-server, multi-client settings. By unifying advanced optimization methods, FedOptimus sets a new standard for efficient and scalable Vertical Federated Learning frameworks, paving the way for broader adoption in complex, privacy-sensitive domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑