This paper accelerates video perception, such as semantic segmentation and human pose estimation, by levering cross-frame redundancies. Unlike the existing approaches, which avoid redundant computations by warping the past features using optical-flow or by performing sparse convolutions on frame differences, we approach the problem from a new perspective: low-bit quantization. We observe that residuals, as the difference in network activations between two neighboring frames, exhibit properties that make them highly quantizable. Based on this observation, we propose a novel quantization scheme for video networks coined as Residual Quantization. ResQ extends the standard, frame-by-frame, quantization scheme by incorporating temporal dependencies that lead to better performance in terms of accuracy vs. bit-width. Furthermore, we extend our model to dynamically adjust the bit-width proportional to the amount of changes in the video. We demonstrate the superiority of our model, against the standard quantization and existing efficient video perception models, using various architectures on semantic segmentation and human pose estimation benchmarks.
In this paper, we propose a preprocessing method for conventional image and video encoders that can make these existing encoders content-aware. By going through our process, a higher quality parameter could be set on a traditional encoder without increasing the output size. A still frame or an image will firstly go through an object detector. Either the properties of the detection result will decide the parameters of the following procedures, or the system will be bypassed if no object is detected in the given frame. The processing method utilizes an adaptive quantization process to determine …
Scene-Aware Vectorized Memory Multi-Agent Framework with Cross-Modal Differentiated Quantization VLMs for Visually Impaired Assistance
School of Information；Software Engineering, University of Electronic Science；Sichuan Provincial Key Laboratory for Human Disease Gene Study, Sichuan Academy of Medical Sciences \& Sichuan Provincial People's Hospital, University of Electronic Science；Faculty of Computing, Harbin Institute of Technology, Harbin, China
Enhancing Predictability of Multi-Tenant DNN Inference for Autonomous Vehicles' Perception
Department of Electrical Engineering and Computer Science, University of Michigan（电气工程与计算机科学系，密歇根大学）；Department of Computer Science and Engineering, Yonsei University（计算机科学与工程系，延世大学）；Department of Computer and Information Sciences, University of Delaware（计算机与信息科学系，特拉华大学）
