In this paper, we tackle a significant challenge in PCA: heterogeneity. When data are collected from different sources with heterogeneous trends while still sharing some congruency, it is critical to extract shared knowledge while retaining the unique features of each source. To this end, we propose personalized PCA (PerPCA), which uses mutually orthogonal global and local principal components to encode both unique and shared features. We show that, under mild conditions, both unique and shared features can be identified and recovered by a constrained optimization problem, even if the covariance matrices are immensely different. Also, we design a fully federated algorithm inspired by distributed Stiefel gradient descent to solve the problem. The algorithm introduces a new group of operations called generalized retractions to handle orthogonality constraints, and only requires global PCs to be shared across sources. We prove the linear convergence of the algorithm under suitable assumptions. Comprehensive numerical experiments highlight PerPCA's superior performance in feature extraction and prediction from heterogeneous datasets. As a systematic approach to decouple shared and unique features from heterogeneous datasets, PerPCA finds applications in several tasks, including video segmentation, topic extraction, and feature clustering.
Optimal Personalized Subspace Learning for Multi-view Tensor Observations
Institute for Financial Studies Shandong University（山东大学金融研究院）；School of Statistics and Data Science Southeast University（东南大学统计与数据科学学院）；Department of Statistics and Applied Probability National University of Singapore（新加坡国立大学统计与应用概率系）
Forecasting Binary Economic Events in Modern Mercantilism: Traditional methodologies coupled with PCA and K-means Quantitative Analysis of Qualitative Sentimental Data
Efficient Personalized Federated PCA with Manifold Optimization for IoT Anomaly Detection
School of Mechatronic Engineering and Automation, Shanghai University（机电工程与自动化学院，上海大学）；School of Intelligent Systems Engineering, Sun Yat-sen University（智能系统工程学院，中山大学）
FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decoupling
Beijing Key Laboratory of MEMS Technology and Device Reliability for Industrial Internet, the School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications（北京工业互联网MEMS技术与器件可靠性重点实验室，智能工程与自动化学院，北京邮电大学）；School of Business Administration, Southwestern University of Finance and Economics（西南财经大学商学院）；Institute of Artificial Intelligence and the School of Computer Science, Beihang University（北京航空航天大学人工智能研究院和计算机学院）；School of Electronic Engineering, Dublin City University（都柏林城市大学电子工程学院）；DreamSoul
Towards Optimal Customized Architecture for Heterogeneous Federated Learning with Contrastive Cloud-Edge Model Decoupling
Southwestern University of Finance and Economics（西南财经大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；City University of Hong Kong（香港城市大学）
FlatLand: Personalized Graph Federated Learning via Tailored Lorentz Space
The Chinese University of Hong Kong（香港中文大学）；Yale University（耶鲁大学）；Huawei Technologies Co., Ltd.（华为技术有限公司）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Brunel University London（伦敦布鲁内尔大学）；Technical University of Darmstadt（达姆施塔特工业大学）；American International University-Bangladesh（孟加拉国国际大学）；Dhaka University of Engineering and Technology（达卡工程技术大学）；George Mason University（乔治梅森大学）
Hypernetworks for Model-Heterogeneous Personalized Federated Learning
the Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China)（青岛软件研究所、计算机科学与技术学院、中国石油大学（华东））；the Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software（山东省智能石油与天然气工业软件重点实验室）；National University of Singapore（新加坡国立大学）
Towards Unified Modeling in Federated Multi-Task Learning via Subspace Decoupling
School of Computer Science, Wuhan University（武汉大学计算机学院）；Faculty of Artificial Intelligence in Education, Central China Normal University（中央财经大学教育人工智能学院）
