Federated Learning (FL), Split Learning (SL), and SplitFed Learning (SFL) are three recent developments in distributed machine learning that are gaining attention due to their ability to preserve the privacy of raw data. Thus, they are widely applicable in various domains where data is sensitive, such as large-scale medical image classification, internet-of-medical-things, and cross-organization phishing email detection. SFL is developed on the confluence point of FL and SL. It brings the best of FL and SL by providing parallel client-side machine learning model updates from the FL paradigm and a higher level of model privacy (while training) by splitting the model between the clients and server coming from SL. However, SFL has communication and computation overhead at the client-side due to the requirement of client-side model synchronization. For the resource-constrained client-side, removal of such requirements is required to gain efficiency in the learning. In this regard, this paper studies SFL without client-side model synchronization. The resulting architecture is known as Multi-head Split Learning. Our empirical studies considering the ResNet18 model on MNIST data under IID data distribution among distributed clients find that Multi-head Split Learning is feasible. Its performance is comparable to the SFL. Moreover, SFL provides only 1%-2% better accuracy than Multi-head Split Learning on the MNIST test set. To further strengthen our results, we study the Multi-head Split Learning with various client-side model portions and its impact on the overall performance. To this end, our results find a minimal impact on the overall performance of the model.
FedImpro: Measuring and Improving Client Update in Federated Learning
Hong Kong Baptist University（香港浸会大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；University of Science and Technology of China（中国科学技术大学）；The University of Sydney（悉尼大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Evaluation and Optimization of Distributed Machine Learning Techniques for Internet of Things
Data61, CSIRO（Data61 联邦科学与工业研究组织）；Sungkyunkwan University（成均馆大学）；Cyber Security Cooperative Research Centre（网络安全合作研究中心）
联邦学习（FL）和分裂学习（SL）是最先进的分布式机器学习技术，用于在不访问客户端或终端设备上的原始数据的情况下实现机器学习训练。然而，它们的\emph{comparative training performance}在现实世界资源受限的物联网（IoT）设备设置下，例如，Raspberry Pi几乎没有被研究过，据我们所知，它还没有被评估和比较，这给从业者带来了不便。这项工作首先提供了FL和SL在现实世界物联网环境中的经验比较，涉及（i）异构数据分布的学习性能和（ii）设备上的执行开销。我们在这项工作中的分析表明，SL的学习性能优于FL下的不平衡的数据分布，但比FL下的极端非IID数据分布差。最近，FL和SL被组合以形成分裂式学习（SFL），以利用它们的每个优点（例如，FL的并行训练和SL的轻量级设备上计算要求）。然后，这项工作考虑FL，SL和SFL，并将它们安装在Raspberry Pi设备上以评估它们的性能，包括训练时间，通信开销，功耗和内存使用。除了评估之外，我们还应用了两个优化。首先，我们通过仔细检查服务器端混合类型模型训练的可能性来推广SFL。广义SFL合并了模型训练的顺序（依赖）和并行（独立）过程，因此对于具有大规模物联网设备的系统是有益的，特别是在服务器端操作。其次，我们提出了务实的技术，大大减少了高达四倍的SL和（广义）SFL的通信开销。
Privacy Leakage in Federated Learning in Radiology Reports: A Comparative Evaluation of Tokenizer-Driven Privacy Risks
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Medical Faculty Heidelberg, Heidelberg University（海德堡大学医学院）；Bilkent University（比尔肯特大学）；Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital（海德堡大学医院放射肿瘤学系模式分析与学习组）
