基于多教师知识蒸馏的视频高效时序句子定位
Efficient Temporal Sentence Grounding in Videos with Multi-Teacher Knowledge Distillation
AI总结：
本文提出基于多教师知识蒸馏的高效模型EMTM，通过异构与同构网络迁移知识并利用知识聚合单元与共享编码器策略，在保持高性能的同时显著提升了视频时序句子定位的计算效率。
摘要：
视频时序句子定位（TSGV）旨在从未剪辑视频中检测由自然语言查询描述的事件时间戳。本文讨论了在保持高性能的同时实现TSGV模型高效计算的挑战。大多数现有方法通过额外层和损失精心设计复杂架构以提高准确率，但存在低效和笨重的问题。尽管一些工作注意到了这一点，但它们仅关注特征融合层，难以在整个笨重的网络中享受高速优势。为解决此问题，我们提出了一种基于知识蒸馏的新型高效多教师模型（EMTM），从异构和同构网络中迁移多样化知识。具体而言，我们首先将异构模型的不同输出统一为单一形式。接着，构建知识聚合单元（KAU）以从多个教师获取高质量的综合软标签。随后，KAU模块利用多尺度视频和全局查询信息自适应地确定不同教师的权重。然后提出共享编码器策略，以解决学生浅层难以从教师中受益的问题，其中同构教师与学生协同训练以对齐它们的隐藏状态。在三个流行的TSGV基准上的大量实验结果表明，我们的方法既有效又高效，且没有多余的装饰。
英文摘要：
Temporal Sentence Grounding in Videos (TSGV) aims to detect the event timestamps described by the natural language query from untrimmed videos. This paper discusses the challenge of achieving efficient computation in TSGV models while maintaining high performance. Most existing approaches exquisitely design complex architectures to improve accuracy with extra layers and loss, suffering from inefficiency and heaviness. Although some works have noticed that, they only make an issue of feature fusion layers, which can hardly enjoy the highspeed merit in the whole clunky network. To tackle this problem, we propose a novel efficient multi-teacher model (EMTM) based on knowledge distillation to transfer diverse knowledge from both heterogeneous and isomorphic networks. Specifically, We first unify different outputs of the heterogeneous models into one single form. Next, a Knowledge Aggregation Unit (KAU) is built to acquire high-quality integrated soft labels from multiple teachers. After that, the KAU module leverages the multi-scale video and global query information to adaptively determine the weights of different teachers. A Shared Encoder strategy is then proposed to solve the problem that the student shallow layers hardly benefit from teachers, in which an isomorphic teacher is collaboratively trained with the student to align their hidden states. Extensive experimental results on three popular TSGV benchmarks demonstrate that our method is both effective and efficient without bells and whistles.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
