Minimax optimization has seen a surge in interest with the advent of modern applications such as GANs, and it is inherently more challenging than simple minimization. The difficulty is exacerbated by the training data residing at multiple edge devices or \textit{clients}, especially when these clients can have heterogeneous datasets and local computation capabilities. We propose a general federated minimax optimization framework that subsumes such settings and several existing methods like Local SGDA. We show that naive aggregation of heterogeneous local progress results in optimizing a mismatched objective function -- a phenomenon previously observed in standard federated minimization. To fix this problem, we propose normalizing the client updates by the number of local steps undertaken between successive communication rounds. We analyze the convergence of the proposed algorithm for classes of nonconvex-concave and nonconvex-nonconcave functions and characterize the impact of heterogeneous client data, partial client participation, and heterogeneous local computations. Our analysis works under more general assumptions on the intra-client noise and inter-client heterogeneity than so far considered in the literature. For all the function classes considered, we significantly improve the existing computation and communication complexity results. Experimental results support our theoretical claims.
H2Tune: Federated Foundation Model Fine-Tuning with Hybrid Heterogeneity
School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；School of Computer Science and Technology, Heilongjiang University（黑龙江大学计算机科学与技术学院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；School of Statistics, Renmin University of China（中国人民大学统计学院）；Zhongguancun Laboratory, China（中关村实验室）；School of Computer Science and Technology, Shandong University（山东大学计算机科学与技术学院）；WeBank Co., Ltd, Shenzhen, China（WeBank有限公司，深圳，中国）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）
CDMA: A Practical Cross-Device Federated Learning Algorithm for General Minimax Problems
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Advanced Technology Institute, Zhejiang University（浙江大学先进技术研究院）；Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）
Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
National Key Laboratory of Parallel and Distributed Computing, College of Computer Science and Technology, National University of Defense Technology（国家并行与分布式计算实验室，计算机科学与技术学院，国防科技大学）
Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；Yale University（耶鲁大学）
Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneity
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
联邦学习（FL）是涉及多个客户端的新兴机器学习范例，例如，移动电话设备，具有协作解决由中央服务器协调的机器学习问题的激励。FL是由Konečnet al.和McMahan et al.在2016年提出的，作为传统集中式机器学习的一种可行的隐私保护替代方案，因为通过构造，训练数据点是分散的，并且永远不会由客户端传输到中央服务器。因此，FL在一定程度上减轻了与集中式数据汇集相关的隐私风险。不幸的是，FL的优化面临着集中式优化通常不需要处理的几个特定问题。在这篇论文中，我们确定了这些挑战中的一些，并提出了新的方法和算法来解决这些问题，最终目标是使实际的FL解决方案支持数学上严格的保证。
FedSPM: Routing-Enabled Federated Learning under Dual Heterogeneity via Semiparametric Mixture
Institute of Statistics and Big Data, Renmin University of China（中国人民大学统计与大数据研究院）；Department of Statistics and Actuarial Science, University of Waterloo（滑铁卢大学统计与精算科学系）
Tackling Privacy Heterogeneity in Differentially Private Federated Learning
Department of Information Engineering, The Chinese University of Hong Kong（信息工程系，香港中文大学）；School of Science and Engineering, Shenzhen Institute of Artificial Intelligence and Robotics for Society（科学与工程学院，深圳人工智能与机器人社会研究院）
HASA: Subnet Allocation for Compute-Constrained Model-Heterogeneous Federated Learning
University of Naples "Federico II"（那不勒斯腓特烈二世大学）；Queen Mary University of London（伦敦玛丽女王大学）；Brandenburg University of Technology Cottbus-Senftenberg（勃兰登堡工业大学）；Tallinn University of Technology（塔林理工大学）；University of Zanjan（赞詹大学）
Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
Hong Kong Baptist University（香港浸会大学）；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
FedCCA: Client-Centric Adaptation against Data Heterogeneity in Federated Learning on IoT Devices
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Centre for Learning, Teaching, and Technology, The Education University of Hong Kong（香港教育大学学习、教学与技术中心）
Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction
Duke University（杜克大学）；University of Maryland, College Park（马里兰大学帕克分校）；George Mason University（乔治梅森大学）；Yale University（耶鲁大学）；The State University of New York at Buffalo（纽约州立大学布法罗分校）
WinFLoRA: Incentivizing Client-Adaptive Aggregation in Federated LoRA under Privacy Heterogeneity
RMIT University（皇家墨尔本理工大学）；Huazhong University of Science and Technology（华中科技大学）；CSIRO’s Data61 and Responsible AI Research (RAIR) Centre（澳大利亚CSIRO数据61和负责任的人工智能研究（RAIR）中心）
