The continuous scaling of deep neural networks has fundamentally transformed machine learning, with larger models demonstrating improved performance across diverse tasks. This growth in model size has dramatically increased the computational resources required for the training process. Consequently, distributed approaches, such as Federated Learning and Split Learning, have become essential paradigms for scalable deployment. However, existing Split Learning approaches assume client homogeneity and uniform split points across all participants. This critically limits their applicability to real-world IoT systems where devices exhibit heterogeneity in computational resources. To address this limitation, this paper proposes Hetero-SplitEE, a novel method that enables heterogeneous IoT devices to train a shared deep neural network in parallel collaboratively. By integrating heterogeneous early exits into hierarchical training, our approach allows each client to select distinct split points (cut layers) tailored to its computational capacity. In addition, we propose two cooperative training strategies, the Sequential strategy and the Averaging strategy, to facilitate this collaboration among clients with different split points. The Sequential strategy trains clients sequentially with a shared server model to reduce computational overhead. The Averaging strategy enables parallel client training with periodic cross-layer aggregation. Extensive experiments on CIFAR-10, CIFAR-100, and STL-10 datasets using ResNet-18 demonstrate that our method maintains competitive accuracy while efficiently supporting diverse computational constraints, enabling practical deployment of collaborative deep learning in heterogeneous IoT ecosystems.
HARMONY: Bridging the Personalization-Generalization Gap by Mitigating Representation Skew in Heterogeneous Split Federated Learning
Department of ECE and INMC, Seoul National University, Seoul, South Korea（首尔国立大学电子与信息科学系及网络信息中心，首尔，韩国）；Graduate School of Data Science, Seoul National University, Seoul, South Korea（首尔国立大学数据科学研究生院，首尔，韩国）；Department of Computer Science, University of Colorado Boulder, CO, USA（科罗拉多大学博尔德分校计算机科学系，科罗拉多州，美国）
Department of Computer Science, Control and Management Engineering, Sapienza University of Rome, Rome, Italy（计算机科学与控制管理工程系，罗马萨皮恩扎大学，意大利罗马）；Institute of Information Science and Technologies "Alessandro Faedo" - ISTI-CNR, Pisa, Italy（信息科学与技术研究所 "Alessandro Faedo" - ISTI-CNR，意大利比萨）；Information Engineering Department, University of Pisa, Pisa, Italy（信息工程系，比萨大学，意大利比萨）；CNRS, i3S, Inria（国家科学研究中心 CNRS、i3S、Inria）
Asteroid: Resource-Efficient Hybrid Pipeline Parallelism for Collaborative DNN Training on Heterogeneous Edge Devices
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Data Science and Analytics Thrust, HKUST (GZ)（香港科技大学（广州）数据科学与分析学域）；The Chinese University of Hong Kong（香港中文大学）
COMSPLIT: A Communication-Aware Split Learning Design for Heterogeneous IoT Platforms
Faculty of Technical Sciences, University of Novi Sad（诺维萨德大学技术科学学院）；Institute for Artificial Intelligence Research and Development of Serbia（塞尔维亚人工智能研究与发展研究所）；Science, Technology and Innovation Unit, The Abdus Salam International Centre for Theoretical Physics（阿卜杜勒·萨拉姆国际理论物理中心科学、技术与创新部）
