arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.09396eess.IVcs.CV
收藏

联合传输与去模糊:一种使用事件的语义通信方法

Joint Transmission and Deblurring: A Semantic Communication Approach Using Events

College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息与电子工程学院);School of Electronic Information, Wuhan University(武汉大学电子信息学院)

Pujing Yang, Guangyi Zhang, Yunlong Cai, Lei Yu, Guanding Yu

AI总结:

针对现有JSCC图像传输忽略运动模糊、事件相机数据难高效传输的问题,提出面向去模糊的JSCC框架,联合传输模糊图像与事件的共享及域专属信息,结合多阶段训练,在有限带宽下实现更优的清晰图像重建效果。

摘要:

基于深度学习的联合信源信道编码(JSCC)正逐渐成为实现高效图像传输的一项颇具前景的技术。然而,现有大多数方法都聚焦于清晰图像的传输,忽略了现实世界中存在的各类挑战,比如由相机抖动或快速移动物体引发的运动模糊。运动模糊通常会降低图像质量,使得传输与重建任务更具挑战性。事件相机能够以极低的延迟异步记录像素强度变化,已在运动去模糊任务中展现出巨大潜力。但事件相机生成的海量数据的高效传输仍是一项重大挑战。在本研究中,我们提出了一种全新的JSCC框架,用于模糊图像与事件的联合传输,旨在在有限的信道带宽下实现高质量重建。该方法被设计为面向去模糊任务的JSCC系统。由于RGB相机与事件相机通过不同模态捕获同一场景,它们的输出既包含共享信息,也包含特定领域的信息。为避免重复传输共享信息,我们分别提取并传输二者的共享信息与特定领域信息。在接收端,接收到的信号由去模糊解码器处理以生成清晰图像。此外,我们引入了一种多阶段训练策略来训练所提出的模型。仿真结果表明,我们的方法显著优于现有的基于JSCC的图像传输方案,能够有效解决运动模糊问题。

英文摘要:

Deep learning-based joint source-channel coding (JSCC) is emerging as a promising technology for effective image transmission. However, most existing approaches focus on transmitting clear images, overlooking real-world challenges such as motion blur caused by camera shaking or fast-moving objects. Motion blur often degrades image quality, making transmission and reconstruction more challenging. Event cameras, which asynchronously record pixel intensity changes with extremely low latency, have shown great potential for motion deblurring tasks. However, the efficient transmission of the abundant data generated by event cameras remains a significant challenge. In this work, we propose a novel JSCC framework for the joint transmission of blurry images and events, aimed at achieving high-quality reconstructions under limited channel bandwidth. This approach is designed as a deblurring task-oriented JSCC system. Since RGB cameras and event cameras capture the same scene through different modalities, their outputs contain both shared and domain-specific information. To avoid repeatedly transmitting the shared information, we extract and transmit their shared information and domain-specific information, respectively. At the receiver, the received signals are processed by a deblurring decoder to generate clear images. Additionally, we introduce a multi-stage training strategy to train the proposed model. Simulation results demonstrate that our method significantly outperforms existing JSCC-based image transmission schemes, addressing motion blur effectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑