arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.02574eess.IVcs.CVcs.MM
收藏

跨模态压缩:面向人类可理解的语义压缩

Cross Modal Compression: Towards Human-comprehensible Semantic Compression

Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Peking University(北京大学);Information Technology R&D Innovation Center of Peking University(北京大学信息技术创新研发中心)

Jiguo Li, Chuanmin Jia, Xinfeng Zhang, Siwei Ma, Wen Gao

摘要:

传统的图像/视频压缩的目标是在尽可能高的信号保真度的情况下降低传输/存储成本。然而,随着近年来对机器分析和语义监控的需求不断增加,语义保真度而不是信号保真度正在成为图像/视频压缩中的另一个新兴问题。随着跨模态转换和生成技术的发展,本文提出了一种基于语义的视觉数据压缩框架--跨模态压缩(CMC),用于对高冗余度的视觉数据(如图像、视频等)进行转换。到一个紧凑的,人类可理解的领域(如文本,草图,语义图,属性等),同时保留语义。具体来说,我们首先制定CMC问题作为一个率失真优化问题。其次,我们研究了传统的图像/视频压缩和最近的特征压缩框架之间的关系,显示我们的CMC和这些以前的框架之间的差异。然后,我们提出了一个新的范式CMC证明其有效性。定性和定量的结果表明,我们提出的CMC可以实现令人鼓舞的重建结果与1/4的压缩比,表现出更好的压缩性能比广泛使用的JPEG基线。

英文摘要:

Traditional image/video compression aims to reduce the transmission/storage cost with signal fidelity as high as possible. However, with the increasing demand for machine analysis and semantic monitoring in recent years, semantic fidelity rather than signal fidelity is becoming another emerging concern in image/video compression. With the recent advances in cross modal translation and generation, in this paper, we propose the cross modal compression~(CMC), a semantic compression framework for visual data, to transform the high redundant visual data~(such as image, video, etc.) into a compact, human-comprehensible domain~(such as text, sketch, semantic map, attributions, etc.), while preserving the semantic. Specifically, we first formulate the CMC problem as a rate-distortion optimization problem. Secondly, we investigate the relationship with the traditional image/video compression and the recent feature compression frameworks, showing the difference between our CMC and these prior frameworks. Then we propose a novel paradigm for CMC to demonstrate its effectiveness. The qualitative and quantitative results show that our proposed CMC can achieve encouraging reconstructed results with an ultrahigh compression ratio, showing better compression performance than the widely used JPEG baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑