arXiv 2502.037242025-10-19cs.CVcs.AIcs.HCcs.LGcs.MM

在黑暗中看见：基于知识蒸馏和对比学习的暗视频动作识别师生框架

Seeing in the Dark: A Teacher-Student Framework for Dark Video Action Recognition via Knowledge Distillation and Contrastive Learning

School of Information Technology, Monash University, Malaysia；Faculty of Information Technology, Monash University, Australia；Robotics \& Autonomous Systems Lab, TCS Research, India；Dept of Electrical Engineering, University of Malaya, Malaysia

Sharana Dharshikgan Suresh Dass, Hrishav Bakul Barua, Ganesh Krishnasamy, Raveendran Paramesran, Raphael C. -W. Phan

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对低光视频动作识别，提出 ActLumos 师生框架，通过 DFF、SupCon、自监督预训练与知识蒸馏，将多流能力压缩为单流推理并取得最先进结果。

摘要：

由于能见度下降会妨碍关键的时空细节，暗或低光（曝光不足）视频中的动作识别是一项具有挑战性的任务。本文提出 ActLumos，这是一种师生框架，可在保持多流级别准确率的同时实现单流推理。教师模型接收双流输入，包括原始暗帧和 retinex 增强帧，由权重共享的 R(2+1)D-34 主干网络处理，并通过动态特征融合（Dynamic Feature Fusion，DFF）模块融合；该模块在每个时间步动态重新加权两个流，强调信息量最大的时间片段。教师模型还加入监督对比损失（SupCon），以锐化类别间隔。学生模型共享 R(2+1)D-34 主干网络，但在测试时仅使用暗帧且不进行融合。学生模型首先在两个数据集的暗片段上以无标签方式进行自监督预训练，随后利用来自教师模型的知识蒸馏进行微调，将教师模型的多流知识迁移到单流模型中。在单流推理下，经过蒸馏的学生模型在 ARID V1.0 上达到 96.92%（Top-1）、在 ARID V1.5 上达到 88.27%、在 Dark48 上达到 48.96% 的最先进准确率。消融研究进一步突出了每个组件的单独贡献：教师模型中的 DFF 优于单一或静态融合，知识蒸馏（KD）将这些增益迁移给单流学生模型，而双视角时空 SSL 在不增加推理成本的情况下优于仅空间或仅时间变体。本工作的官方网站位于：https://github.com/HrishavBakulBarua/ActLumos

英文摘要：

Action recognition in dark or low-light (under-exposed) videos is a challenging task due to visibility degradation, which can hinder critical spatiotemporal details. This paper proposes ActLumos, a teacher-student framework that attains single-stream inference while retaining multi-stream level accuracy. The teacher consumes dual stream inputs, which include original dark frames and retinex-enhanced frames, processed by weight-shared R(2+1)D-34 backbones and fused by a Dynamic Feature Fusion (DFF) module, which dynamically re-weights the two streams at each time step, emphasising the most informative temporal segments. The teacher is also included with a supervised contrastive loss (SupCon) that sharpens class margins. The student shares the R(2+1)D-34 backbone but uses only dark frames and no fusion at test time. The student is first pre-trained with self-supervision on dark clips of both datasets without their labels and then fine-tuned with knowledge distillation from the teacher, transferring the teacher's multi-stream knowledge into a single-stream model. Under single-stream inference, the distilled student attains state-of-the-art accuracy of 96.92% (Top-1) on ARID V1.0, 88.27% on ARID V1.5, and 48.96% on Dark48. Ablation studies further highlight the individual contributions of each component, i.e., DFF in the teacher outperforms single or static fusion, knowledge distillation (KD) transfers these gains to the single-stream student, and two-view spatio-temporal SSL surpasses spatial-only or temporal-only variants without increasing inference cost. The official website of this work is available at: https://github.com/HrishavBakulBarua/ActLumos

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。