VA-RED^2: Video Adaptive Redundancy Reduction



Published on



Computer Vision ICLR

Performing inference on deep learning models for videos remains a challenge due to the large amount of computational resources required to achieve robust recognition. An inherent property of real-world videos is the high correlation of information across frames which can translate into redundancy in either temporal or spatial feature maps of the models, or both. The type of redundant features depends on the dynamics and type of events in the video: static videos have more temporal redundancy while videos focusing on objects tend to have more channel redundancy. Here we present a redundancy reduction framework, termed VA-RED^2, which is {\em input-dependent}. Specifically, our VA-RED^2 framework uses an input-dependent policy to decide how many features need to be computed for temporal and channel dimensions. To keep the capacity of the original model, after fully computing the necessary features, we reconstruct the remaining redundant features from those using cheap operations. We learn the adaptive policy jointly with the network weights in a differentiable way with a shared-weight mechanism, making it highly efficient. Extensive experiments on multiple video datasets and two types of video tasks show that our framework achieves 20\% - 40\% reduction in computation (FLOPs) when compared to state-of-the-art methods without any performance loss.

This paper has been published at ICLR 2021

Please cite our work using the BibTeX below.

title={{\{}VA{\}}-{\{}RED{\}}{\$}{\textasciicircum}2{\$}: Video Adaptive Redundancy Reduction},
author={Bowen Pan and Rameswar Panda and Camilo Luciano Fosco and Chung-Ching Lin and Alex J Andonian and Yue Meng and Kate Saenko and Aude Oliva and Rogerio Feris},
booktitle={International Conference on Learning Representations},
Close Modal