BLINC：用于低复杂度帧内编码的轻量级双峰学习VVC
BLINC: Lightweight Bimodal Learning for Low-Complexity VVC Intra Coding
摘要：
最新的视频编码标准，通用视频编码（VVC），实现了几乎两倍的编码效率相比，其前身，高效率视频编码（HEVC）。然而，与HEVC相比，实现这种效率（对于帧内编码）需要31倍的计算复杂性，使得其对于低功率和实时应用具有挑战性。本文提出了一种新的机器学习方法，联合和单独采用两种形式的功能，以简化帧内编码决策。首先，提取一组特征，使用现有的DCT核心的VVC，以评估纹理特征，并形成第一模态的数据。这产生了高质量的功能，几乎没有开销。相邻块处的帧内模式的分布也用于形成数据的第二模态，其提供关于帧的统计信息。其次，设计了一种两步特征约简方法，该方法减小了特征集的大小，使得具有有限数量参数的轻量级模型可以用于学习帧内模式决策任务。第三，提出了三种独立的训练策略：（1）使用第一（单一）数据模态的离线训练策略，（2）使用第二（单一）模态的在线训练策略，以及（3）使用双峰学习的混合在线-离线策略。最后，基于所提出的学习策略，提出了一种低复杂度的编码算法。大量的实验结果表明，所提出的方法可以减少高达24%的编码时间，编码效率的损失可以忽略不计。此外，它演示了如何双峰学习策略可以提高学习的性能。最后，所提出的方法具有非常低的计算开销（0.2%），并且使用VVC编码器的现有组件，这使得它比竞争解决方案更实用。
英文摘要：
The latest video coding standard, Versatile Video Coding (VVC), achieves almost twice coding efficiency compared to its predecessor, the High Efficiency Video Coding (HEVC). However, achieving this efficiency (for intra coding) requires 31x computational complexity compared to HEVC, making it challenging for low power and real-time applications. This paper, proposes a novel machine learning approach that jointly and separately employs two modalities of features, to simplify the intra coding decision. First a set of features are extracted that use the existing DCT core of VVC, to assess the texture characteristics, and forms the first modality of data. This produces high quality features with almost no overhead. The distribution of intra modes at the neighboring blocks is also used to form the second modality of data, which provides statistical information about the frame. Second, a two-step feature reduction method is designed that reduces the size of feature set, such that a lightweight model with a limited number of parameters can be used to learn the intra mode decision task. Third, three separate training strategies are proposed (1) an offline training strategy using the first (single) modality of data, (2) an online training strategy that uses the second (single) modality, and (3) a mixed online-offline strategy that uses bimodal learning. Finally, a low-complexity encoding algorithms is proposed based on the proposed learning strategies. Extensive experimental results show that the proposed methods can reduce up to 24% of encoding time, with a negligible loss of coding efficiency. Moreover, it is demonstrated how a bimodal learning strategy can boost the performance of learning. Lastly, the proposed method has a very low computational overhead (0.2%), and uses existing components of a VVC encoder, which makes it much more practical compared to competing solutions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
