量化-小TinyLLaVA：一种新的多模态基础模型实现了高效的分裂学习
Quantized-Tinyllava: a new multimodal foundation model enables efficient split learning
AI总结：
Quantized-TinyLLaVA通过量化压缩和高效分裂学习框架，在减少通信开销的同时保持模型性能，提升隐私保护能力。
摘要：
多模态基础模型正在越来越多地在金融、生物医学和个人标识等跨领域中训练敏感数据。然而，这种分布式设置由于需要跨分区的数据共享而引发了严重的隐私问题。分裂学习通过在不交换原始数据的情况下实现协作模型训练来解决这些问题，但这也带来了显著的挑战：在分区之间传输高维中间特征表示会导致巨大的通信成本。为了解决这一挑战，我们提出了Quantized-TinyLLaVA，一种具有集成通信高效分裂学习框架的多模态基础模型。我们的方法采用了一个压缩模块，在传输前将中间特征量化为离散表示，从而大幅减少通信开销。此外，我们推导出一种基于熵编码理论的原理性量化策略，以确定最优的离散表示层级数量。我们部署了我们的框架在一个两分区设置中，其中一组作为客户端，另一组作为服务器，以真实模拟分布式训练。在该设置下，Quantized-TinyLLaVA在2位量化下实现了通信开销的近87.5%的减少，同时在五个基准数据集上保持了原始16位模型的性能。此外，我们的压缩表示在对抗特征翻转攻击时表现出增强的鲁棒性，验证了传输的隐私性。代码可在https://github.com/anonymous-1742/Quantized-TinyLLaVA上获得。
英文摘要：
Multimodal foundation models are increasingly trained on sensitive data across domains such as finance, biomedicine, and personal identifiers. However, this distributed setup raises serious privacy concerns due to the need for cross-partition data sharing. Split learning addresses these concerns by enabling collaborative model training without raw data exchange between partitions, yet it introduces a significant challenge: transmitting high-dimensional intermediate feature representations between partitions leads to substantial communication costs. To address this challenge, we propose Quantized-TinyLLaVA, a multimodal foundation model with an integrated communication-efficient split learning framework. Our approach adopts a compression module that quantizes intermediate feature into discrete representations before transmission, substantially reducing communication overhead. Besides, we derive a principled quantization strategy grounded in entropy coding theory to determine the optimal number of discrete representation levels. We deploy our framework in a two-partition setting, with one partition operating as the client and the other as the server, to realistically simulate distributed training. Under this setup, Quantized-TinyLLaVA achieves an approximate \textbf{87.5\%} reduction in communication overhead with 2-bit quantization, while maintaining performance of the original 16-bit model across five benchmark datasets. Furthermore, our compressed representations exhibit enhanced resilience against feature inversion attacks, validating the privacy of transmission. The code is available at https://github.com/anonymous-1742/Quantized-TinyLLaVA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
