Federated fine-tuning offers a promising solution for adapting Large Language Models (LLMs) to downstream tasks while safeguarding data privacy. However, its high computational and communication demands hinder its deployment on resource-constrained devices. In this paper, we propose SmartFed, a resource-efficient federated fine-tuning framework. SmartFed intelligently reuses knowledge embedded in existing LoRA modules, eliminating the need for expensive training from scratch when adapting LLMs to new tasks. To effectively exploit this knowledge and ensure scalability, we introduce the Mixture of Rank-Wise Experts (MoRE). MoRE decomposes LoRA modules into fine-grained rank-level experts. These experts are selectively activated and combined based on input semantics and resource budgets. Moreover, to optimize resource utilization, we present the Elastic Expert Quota Allocation (EEQA). EEQA adaptively allocates expert capacity across parameter matrices based on their contribution to model performance, focusing computing resources on the critical experts. Extensive evaluations across multiple benchmarks demonstrate that SmartFed significantly outperforms existing methods in model performance and training efficiency.
Federated Spatial Reuse Optimization in Next-Generation Decentralized IEEE 802.11 WLANs
CTTC(加泰罗尼亚电信技术中心);Trinity College Dublin(都柏林圣三一学院);Imperial College London(伦敦帝国理工学院);New York University(纽约大学);University of Modena and Reggio Emilia(摩德纳和雷焦艾米利亚大学);Xiamen University(厦门大学);Universitat Pompeu Fabra(庞培法布拉大学)
随着无线标准的发展,引入了更复杂的功能来满足吞吐量、延迟、安全性和效率方面不断增长的要求。为了释放这些新功能的潜力,目前正在利用人工智能(AI)和机器学习(ML)从数据中导出模型和协议,而不是通过手工编程。在本文中,我们探讨了在下一代无线局域网(WLAN)中应用ML的可行性。更具体地说,我们专注于IEEE 802.11ax空间复用(SR)问题,并通过联邦学习(FL)模型预测其性能。本工作中概述的FL解决方案集是2021年国际电信联盟(ITU)AI for 5G挑战赛的一部分。
FedAPTA: Federated Multi-task Learning for Heterogeneous Devices with Adaptive Layer-wise Pruning and Task-aware Aggregation
Soochow University(苏州大学);Key Laboratory of General Artificial Intelligence and Large Models in Provincial Universities (Soochow University)(江苏省普通高校通用人工智能与大模型重点实验室(苏州大学));China Mobile (Suzhou) Software Technology Co., Ltd.(中国移动(苏州)软件技术有限公司)