arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.03377cs.CV
收藏

用于在线动作检测的长短时限Transformer

Long Short-Term Transformer for Online Action Detection

Amazon/AWS AI(亚马逊/AWS 人工智能)

Mingze Xu, Yuanjun Xiong, Hao Chen, Xinyu Li, Wei Xia, Zhuowen Tu, Stefano Soatto

摘要:

我们提出了长短期Transformer(LSTR),时间建模算法的在线动作检测,它采用了长期和短期记忆机制来建模延长序列数据。它由LSTR编码器组成,该LSTR编码器动态地利用来自扩展时间窗口的粗尺度历史信息(例如,跨度达8分钟的2048个帧),以及聚焦于短时间窗口的LSTR解码器(例如,32帧,跨度8秒),以模拟数据的精细尺度特征。与以往的工作相比,LSTR提供了一种有效的和高效的方法来建模长视频与更少的统计,这是通过广泛的实证分析验证。LSTR在三个标准的在线动作检测基准测试中达到了最先进的性能,THUMOS'14,TVSeries和HACS Segment。代码已在https://xumingze0308.github.io/projects/lstr上提供

英文摘要:

We present Long Short-term TRansformer (LSTR), a temporal modeling algorithm for online action detection, which employs a long- and short-term memory mechanism to model prolonged sequence data. It consists of an LSTR encoder that dynamically leverages coarse-scale historical information from an extended temporal window (e.g., 2048 frames spanning of up to 8 minutes), together with an LSTR decoder that focuses on a short time window (e.g., 32 frames spanning 8 seconds) to model the fine-scale characteristics of the data. Compared to prior work, LSTR provides an effective and efficient method to model long videos with fewer heuristics, which is validated by extensive empirical analysis. LSTR achieves state-of-the-art performance on three standard online action detection benchmarks, THUMOS'14, TVSeries, and HACS Segment. Code has been made available at: https://xumingze0308.github.io/projects/lstr

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑