在测试集上对率失真曲线求平均以比较学习型视频编解码器的做法可能导致误导性结论
The Practice of Averaging Rate-Distortion Curves over Testsets to Compare Learned Video Codecs Can Cause Misleading Conclusions
AI总结：
本文证明在测试集上平均率失真曲线会误导学习型视频编解码器的性能评估，建议社区报告每序列RD曲线并基于每序列指标平均值计算性能，以确保比较公平准确。
摘要：
本文旨在证明，学习型视频压缩社区普遍存在的在测试视频集上对率失真（RD）曲线求平均的做法，如何在评估编解码器性能时导致误导性结论。通过对一个简单案例的解析分析以及两种最新学习型视频编解码器的实验结果，我们展示了平均RD曲线如何误导对不同编解码器的比较评估，特别是当数据集中的视频具有不同特征和工作范围时。我们说明了一个与测试集其余部分具有不同RD特征的视频如何不成比例地影响平均RD曲线，可能掩盖编解码器在大多数单个序列上的优越性能。以两种最新学习型视频编解码器在UVG数据集上的应用为案例研究，我们证明从平均RD曲线计算性能指标（如BD rate）所得出的结论，与从计算每序列指标平均值所得出的结论相矛盾。因此，我们认为学习型视频压缩社区也应报告每序列RD曲线，并且测试集的性能指标应从每序列指标的平均值中计算，类似于传统视频编码中的既定做法，以确保编解码器比较的公平和准确。
英文摘要：
This paper aims to demonstrate how the prevalent practice in the learned video compression community of averaging rate-distortion (RD) curves across a test video set can lead to misleading conclusions in evaluating codec performance. Through analytical analysis of a simple case and experimental results with two recent learned video codecs, we show how averaged RD curves can mislead comparative evaluation of different codecs, particularly when videos in a dataset have varying characteristics and operating ranges. We illustrate how a single video with distinct RD characteristics from the rest of the test set can disproportionately influence the average RD curve, potentially overshadowing a codec's superior performance across most individual sequences. Using two recent learned video codecs on the UVG dataset as a case study, we demonstrate computing performance metrics, such as the BD rate, from the average RD curve suggests conclusions that contradict those reached from calculating the average of per-sequence metrics. Hence, we argue that the learned video compression community should also report per-sequence RD curves and performance metrics for a test set should be computed from the average of per-sequence metrics, similar to the established practice in traditional video coding, to ensure fair and accurate codec comparisons.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
