AI 中文总结
针对极端低带宽丢包网络的语义通信问题，提出ResiGLC框架，结合语言模型上下文预测与掩码学习，通过渐进式解码提升抗丢包性，可在低带宽下改善通信的感知质量。
AI 中文摘要
在极端低带宽网络场景中，生成式语义编解码器已成为降低视觉通信带宽成本的有前景解决方案。然而，这些学习得到的编解码器通常仅针对压缩效率进行优化，因此对传输错误缺乏鲁棒性。在这些高度紧凑的生成式隐表示中，丢包导致的损坏会导致保真度和真实感出现更严重的下降，且这种下降因隐上下文间的严重错误传播以及多步解码过程而加剧。在本文中，我们提出了ResiGLC，一种专为极端低带宽丢包网络下的鲁棒语义通信设计的新型抗丢包生成式隐编码框架。受生成与压缩之间固有的目标一致性的启发，我们充分利用了语言模型出色的上下文预测能力。结合掩码学习策略，我们的模型支持隐码的任意上下文建模，这可以缓解错误传播并处理不可预测的丢包模式。在接收端，我们提出了一种渐进式抗丢包解码流水线，该流水线分别利用了隐码的上下文关系和生成式隐空间中的多模态语义先验。通过针对压缩效率和抗丢包性进行联合优化，我们提出的渐进式解码机制在处理动态丢包时表现出平稳的性能下降。通过广泛的实验评估，我们证实，在丢包网络条件下，ResiGLC能够在极端低带宽成本下，有效提升感知保真度和真实感方面的抗丢包能力。
英文摘要
In extreme-low bandwidth network scenarios, generative semantic codecs have emerged as promising solutions to reduce bandwidth cost for visual communications. However, these learned codecs are usually optimized solely for compression efficiency and thus not robust against transmission errors. Corruptions due to packet-loss among these highly compact generative latent representations often cause more critical degradation in fidelity and realism, intensified by the severe error propagation across the latent contexts and multi-step decoding process. In this paper, we propose ResiGLC, a novel loss-resilient generative latent coding framework designed for robust semantic communication over extreme-low bandwidth packet-loss networks. Motivated by the inherent goal-consistency between generation and compression, we sufficiently exploit the impressive in-context predictive capabilities of language models. Integrated with the masked learning strategy, our model supports arbitrary context modeling of latent codes, which could mitigate the error propagation and handle unpredictable packet loss patterns. At the receiver, a progressive resilient decoding pipeline is presented, which leverages both the contextual relationship of the latent codes and the multi-modal semantic prior in the generative latent space, separately. By jointly optimizing toward both compression efficiency and packet-loss resilience, our proposed progressive decoding mechanism offers graceful performance when dealing with dynamic packet losses. Through extensive experimental evaluations, we establish that under packet-loss network conditions, ResiGLC can effectively improve the loss-resilience in terms of perceptual fidelity and realism qualities with extreme-low bandwidth cost.
CommentsAccepted to appear in IEEE TMC. 14 pages, 15 figures, and 5 tables
