An important aspect of summarizing videos is understanding the temporal context behind each part of the video to grasp what is and is not important. Video summarization models have in recent years modeled spatio-temporal relationships to represent this information. These models achieved state-of-the-art correlation scores on important benchmark datasets. However, what has not been reviewed is whether spatio-temporal relationships are even required to achieve state-of-the-art results. Previous work in activity recognition has found biases, by prioritizing static cues such as scenes or objects, over motion information. In this paper we inquire if similar spurious relationships might influence the task of video summarization. To do so, we analyse the role that temporal information plays on existing benchmark datasets. We first estimate a baseline with temporally invariant models to see how well such models rank on benchmark datasets (TVSum and SumMe). We then disrupt the temporal order of the videos to investigate the impact it has on existing state-of-the-art models. One of our findings is that the temporally invariant models achieve competitive correlation scores that are close to the human baselines on the TVSum dataset. We also demonstrate that existing models are not affected by temporal perturbations. Furthermore, with certain disruption strategies that shuffle fixed time segments, we can actually improve their correlation scores. With these results, we find that spatio-temporal relationship play a minor role and we raise the question whether these benchmarks adequately model the task of video summarization. Code available at: https://github.com/AashGan/TemporalPerturbSum
A Unified Framework for Generic, Query-Focused, Privacy Preserving and Update Summarization using Submodular Information Measures
Department of Computer Science, Indian Institute of Technology Bombay（印度理工学院孟买分校计算机科学系）；Department of Computer Science, University of Texas at Dallas（得克萨斯大学达拉斯分校计算机科学系）；Department of Electrical and Computer Engineering, University of Washington, Seattle（华盛顿大学西雅图分校电气与计算机工程系）；School of Technology and Computer Science, Tata Institute of Fundamental Research, Mumbai（塔塔基础研究所孟买技术与计算机科学学院）
Using 3D Convolutional Neural Networks to Learn Spatiotemporal Features for Automatic Surgical Gesture Recognition in Video
National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）；University Hospital Carl Gustav Carus（卡尔·古斯塔夫·卡鲁斯大学医院）；TU Dresden（德累斯顿工业大学）；Centre for Tactile Internet with Human-in-the-Loop (CeTI)（人在环触觉互联网中心）
A 4D Representation for Training-Free Agentic Reasoning from Monocular Laparoscopic Video
Computer Aided Medical Procedures, TU Munich（慕尼黑工业大学计算机辅助医疗程序研究所）；Hospital of the LMU Munich, Ludwig-Maximilians-Universität (LMU)（慕尼黑大学医院）；Munich Center for Machine Learning（慕尼黑机器学习中心）
