通过稳定客户端聚类的并发分割学习
Concurrent Split Learning Through Stable Client Clustering
- RWTH Aachen University（亚琛工业大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对固定全局批次限制客户端参与的问题，提出全局聚类并行分割学习（GCPSL），通过稳定客户端聚类并发执行多个分割学习工作负载，在四GPU上显著缩短训练时间，并平衡参与度与精度。
AI中文摘要：
使用固定的全局批次进行训练限制了在任何一步中能够提供示例的分布式客户端的数量。我们研究了一种在不增加单个工作负载所处理批次的情况下利用额外服务器工作器的方法。全局聚类并行分割学习（GCPSL）将客户端分配到固定聚类中，为每个聚类并发执行一个带有全局采样的并行分割学习（GPSL）工作负载，并定期融合客户端和服务器模型段。在256个逻辑客户端的模拟中，将群体分配到更多工作负载中提高了直接数据参与度，而较小的聚类可能会产生精度成本。一个四H100实现的标签感知GCPSL在三次匹配运行中，在$6.13 \pm 0.15$分钟内达到85%的CIFAR-10验证精度，而相同工作负载串行化时需要$19.09 \pm 0.45$分钟。在四GPU分配内，大小平衡和随机固定关联在相似的平均时间内达到目标（5.70和5.66分钟）；大小平衡将直接参与度提高了3.25个百分点。这些测量表征了稳定客户端分割学习中执行并发性、分配信息、参与度和精度之间的权衡。
英文摘要：
Training with a fixed global batch limits how many distributed clients can provide examples in any one step. We examine a way to use additional server workers without increasing the batch processed by an individual workload. Global Clustered Parallel Split Learning (GCPSL) assigns clients to fixed clusters, executes a Parallel Split Learning with Global Sampling (GPSL) workload for each cluster concurrently, and periodically fuses the client and server model segments. In simulations with 256 logical clients, dividing the population across more workloads improves direct data participation, while smaller clusters can incur an accuracy cost. A four-H100 implementation of label-aware GCPSL reaches 85% CIFAR-10 validation accuracy in $6.13 \pm 0.15$ minutes over three matched runs, versus $19.09 \pm 0.45$ minutes when the same workloads are serialized. Within the four-GPU allocation, size-balanced and random fixed affiliations reach the target in similar mean times (5.70 and 5.66 minutes); size balancing increases direct participation by 3.25 percentage points. These measurements characterize a trade-off among execution concurrency, assignment information, participation, and accuracy for stable-client split learning.
