arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.16670cs.CVeess.IV
收藏

面向VCM的端到端可学习多尺度特征压缩

End-to-End Learnable Multi-Scale Feature Compression for VCM

Kyung Hee University(庆熙大学);Electronics and Telecommunications Research Institute(韩国电子通信研究院)

Yeongwoong Kim, Hyewon Jeong, Janghyun Yu, Younhee Kim, Jooyoung Lee, Se Yoon Jeong, Hui Yong Kim

AI总结:

本文提出一种端到端可学习的多尺度特征压缩方法,通过交织融合与编码过程消除特征冗余,在目标检测中实现至少52%的BD-rate降低并将编码时间减少5至27倍。

摘要:

基于深度学习的机器视觉应用的激增催生了一种新型压缩方式,即面向机器的视频编码(VCM)。VCM与传统视频编码的不同之处在于,它针对机器视觉性能而非人类视觉质量进行优化。在MPEG-VCM的特征压缩赛道中,从图像中提取的多尺度特征需要进行压缩。近期的特征压缩研究表明,基于多功能视频编码(VVC)标准的方法相较于MPEG-VCM特征锚点可实现高达96%的BD-rate降低。然而,该方法仍非最优,因为VVC并非为提取特征而设计,而是为自然图像设计。此外,VVC的高编码复杂度使得在不牺牲性能的前提下设计轻量级编码器变得困难。为应对这些挑战,我们提出了一种新颖的多尺度特征压缩方法,该方法既能对提取特征进行端到端优化,又能实现轻量级编码器的设计。所提模型将可学习压缩器与多尺度特征融合网络相结合,从而有效消除多尺度特征中的冗余。我们没有简单地级联融合网络和压缩网络,而是以交织方式集成融合与编码过程。我们的模型首先对较大尺度的特征进行编码以获得潜在表示,然后将该潜在表示与较小尺度的特征进行融合。此过程依次执行,直到最小尺度的特征完成融合,最终阶段的编码潜在表示再进行熵编码以用于传输。结果表明,我们的模型在目标检测任务中相较于先前方法至少实现了52%的BD-rate降低,并且编码时间减少了5倍至27倍。

英文摘要:

The proliferation of deep learning-based machine vision applications has given rise to a new type of compression, so called video coding for machine (VCM). VCM differs from traditional video coding in that it is optimized for machine vision performance instead of human visual quality. In the feature compression track of MPEG-VCM, multi-scale features extracted from images are subject to compression. Recent feature compression works have demonstrated that the versatile video coding (VVC) standard-based approach can achieve a BD-rate reduction of up to 96% against MPEG-VCM feature anchor. However, it is still sub-optimal as VVC was not designed for extracted features but for natural images. Moreover, the high encoding complexity of VVC makes it difficult to design a lightweight encoder without sacrificing performance. To address these challenges, we propose a novel multi-scale feature compression method that enables both the end-to-end optimization on the extracted features and the design of lightweight encoders. The proposed model combines a learnable compressor with a multi-scale feature fusion network so that the redundancy in the multi-scale features is effectively removed. Instead of simply cascading the fusion network and the compression network, we integrate the fusion and encoding processes in an interleaved way. Our model first encodes a larger-scale feature to obtain a latent representation and then fuses the latent with a smaller-scale feature. This process is successively performed until the smallest-scale feature is fused and then the encoded latent at the final stage is entropy-coded for transmission. The results show that our model outperforms previous approaches by at least 52% BD-rate reduction and has $\times5$ to $\times27$ times less encoding time for object detection...

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑