CoDecide:面向高吞吐量视频分析的内容自适应视频解码
CoDecide: Content-Adaptive Video Decoding for High-Throughput Video Analytics
AI总结:
CoDecide是一种仅对含新内容的帧解码的回溯分析系统,用DINOv2监督压缩域学生模型区分新内容,在多数据集上比逐帧检测流水线快15-24倍,检测性能优于同类压缩域选择器。
摘要:
视频档案分析受限于解码过程。在对每一帧运行检测器的密集流水线中,解码仅占运行时间的一小部分,但一旦检测器被限制在选定帧上,解码就会成为主要成本。我们推出CoDecide,这是一种仅对出现新内容的帧进行解码的回溯分析系统,利用压缩比特流中的信号来定位这些帧。运动信息能告知我们有物体在移动,但无法区分该物体是场景中的新物体还是一直存在的物体。为了做出这种区分,我们用冻结的DINOv2教师模型生成的无标签新奇信号来监督轻量级的压缩域学生模型,该信号仅在语义上新的区域触发,而非任何运动区域。因此,该学生模型可泛化到新的摄像头部署场景,无需针对每个摄像头进行重新训练。在推理阶段,它会决定哪些帧需要解码、哪些区域需要重建。解码帧的检测结果会通过运动向量传播到被跳过的帧。我们采用留一数据集法协议,在VIRAT、MEVA和ShanghaiTech数据集上对CoDecide进行评估。与密集的逐帧检测器相比,在相同的解码预算下,CoDecide能比现有压缩域选择器恢复更多的移动物体检测结果,甚至优于直接在目标摄像头训练的竞争对手。CoDecide的运行速度比对每一帧进行解码和检测的流水线快15至24倍。代码可在该https URL获取。
英文摘要:
Analytics on video archives are bottlenecked by decoding. In a dense pipeline that runs a detector on every frame, decoding is a small fraction of the runtime, but once the detector is restricted to selected frames, decoding takes over as the main cost. We introduce CoDecide, a retrospective analytics system that only decodes the frames where new content appears, using signals from the compressed bitstream to find them. Motion tells us something is moving, but not whether it is new to the scene or has been there the whole time. To make that distinction, we supervise a lightweight compressed-domain student on a label-free novelty signal from a frozen DINOv2 teacher, which fires on semantically new regions rather than on any motion. Consequently, the student model generalizes to new camera deployments without requiring per-camera retraining. At inference, it decides which frames to decode and which regions to reconstruct. Detections from the decoded frames are propagated to the skipped frames using motion vectors. We evaluate CoDecide with a leave-one-dataset-out protocol on VIRAT, MEVA, and ShanghaiTech. Against a dense per-frame detector, CoDecide recovers more moving-object detections than prior compressed-domain selectors at an equivalent decoding budget, outperforming even a competitor trained directly on the target camera. CoDecide runs 15 to 24x faster than a pipeline that decodes and detects every frame. Code is available at https://github.com/shristy51/CoDecide.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。