面向机器的视频编码：一种协同压缩与智能分析的范式
Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
AI总结：
本文提出面向机器的视频编码（VCM）新范式，结合AI技术弥合机器视觉特征编码与人类视觉视频编码的鸿沟，梳理相关技术进展并提出解决方案，初步验证了性能与效率提升。
摘要：
旨在压缩并重建整帧的视频编码，与仅保留并传输最关键信息的特征压缩，处于天平的两端。也就是说，前者具备紧凑性与高效性以服务于机器视觉，后者则具备完整保真度以契合人类感知。近期，视频压缩领域的前沿趋势（例如基于深度学习的编码工具、端到端图像/视频编码）以及MPEG-7紧凑特征描述符标准（即Compact Descriptors for Visual Search和Compact Descriptors for Video Analysis），分别在各自方向上推动了可持续的快速发展。在本文中，得益于预测模型、生成模型等蓬勃发展的人工智能技术，我们对由新兴MPEG标准化工作催生的新领域——面向机器的视频编码（Video Coding for Machines，VCM）展开了探索。面向协同压缩与智能分析的目标，VCM试图弥合面向机器视觉的特征编码与面向人类视觉的视频编码之间的鸿沟。结合兴起的“先分析后压缩”实例Digital Retina，本文首先给出了VCM的定义、公式表述与范式。同时，我们从MPEG标准化的独特视角，系统梳理了视频压缩与特征压缩领域的最新技术，为在各类人工智能应用中实现视频流与特征流的协同压缩提供了学术与工业层面的依据。最后，我们提出了潜在的VCM解决方案，初步结果验证了其性能与效率提升，本文还对未来方向进行了探讨。
英文摘要：
Video coding, which targets to compress and reconstruct the whole frame, and feature compression, which only preserves and transmits the most critical information, stand at two ends of the scale. That is, one is with compactness and efficiency to serve for machine vision, and the other is with full fidelity, bowing to human perception. The recent endeavors in imminent trends of video compression, e.g. deep learning based coding tools and end-to-end image/video coding, and MPEG-7 compact feature descriptor standards, i.e. Compact Descriptors for Visual Search and Compact Descriptors for Video Analysis, promote the sustainable and fast development in their own directions, respectively. In this paper, thanks to booming AI technology, e.g. prediction and generation models, we carry out exploration in the new area, Video Coding for Machines (VCM), arising from the emerging MPEG standardization efforts1. Towards collaborative compression and intelligent analytics, VCM attempts to bridge the gap between feature coding for machine vision and video coding for human vision. Aligning with the rising Analyze then Compress instance Digital Retina, the definition, formulation, and paradigm of VCM are given first. Meanwhile, we systematically review state-of-the-art techniques in video compression and feature compression from the unique perspective of MPEG standardization, which provides the academic and industrial evidence to realize the collaborative compression of video and feature streams in a broad range of AI applications. Finally, we come up with potential VCM solutions, and the preliminary results have demonstrated the performance and efficiency gains. Further direction is discussed as well.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
