NSC-SL：一种带宽感知的神经子空间压缩用于通信高效的分裂学习
NSC-SL: A Bandwidth-Aware Neural Subspace Compression for Communication-Efficient Split Learning
AI总结：
NSC-SL通过带宽感知的自适应压缩算法，实现通信高效的分裂学习，有效减少通信开销并保持模型收敛所需的语义信息。
摘要：
神经网络规模的扩大对分布式机器学习构成了重大挑战，尤其是在通信资源有限的情况下。虽然分裂学习（SL）通过将模型层分布在客户端和服务器之间来减轻客户端的计算负担，但频繁传输中间激活和梯度导致了显著的通信开销。为了解决这个问题，我们提出了NSC-SL，一种带宽感知的自适应压缩算法，用于通信高效的SL。NSC-SL首先根据奇异值分布动态确定低秩近似最优秩，以适应实时带宽约束。然后，NSC-SL通过交替正交迭代与残差反馈进行误差补偿的张量分解，有效最小化截断损失。协作机制使NSC-SL能够在保持对收敛至关重要的语义丰富信息的同时实现高压缩比。广泛实验展示了NSC-SL出色的性能。
英文摘要：
The expanding scale of neural networks poses a major challenge for distributed machine learning, particularly under limited communication resources. While split learning (SL) alleviates client computational burden by distributing model layers between clients and server, it incurs substantial communication overhead from frequent transmission of intermediate activations and gradients. To tackle this issue, we propose NSC-SL, a bandwidth-aware adaptive compression algorithm for communication-efficient SL. NSC-SL first dynamically determines the optimal rank of low-rank approximation based on the singular value distribution for adapting real-time bandwidth constraints. Then, NSC-SL performs error-compensated tensor factorization using alternating orthogonal iteration with residual feedback, effectively minimizing truncation loss. The collaborative mechanisms enable NSC-SL to achieve high compression ratios while preserving semantic-rich information essential for convergence. Extensive experiments demonstrate the superb performance of NSC-SL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
