Online action detection (OAD) aims to identify ongoing actions from streaming video in real-time, without access to future frames. Since these actions manifest at varying scales of granularity, ranging from coarse to fine, projecting an entire set of action frames to a single latent encoding may result in a lack of local information, necessitating the acquisition of action features across multiple scales. In this paper, we propose a multi-scale action learning transformer (MALT), which includes a novel recurrent decoder (used for feature fusion) that includes fewer parameters and can be trained more efficiently. A hierarchical encoder with multiple encoding branches is further proposed to capture multi-scale action features. The output from the preceding branch is then incrementally input to the subsequent branch as part of a cross-attention calculation. In this way, output features transition from coarse to fine as the branches deepen. We also introduce an explicit frame scoring mechanism employing sparse attention, which filters irrelevant frames more efficiently, without requiring an additional network. The proposed method achieved state-of-the-art performance on two benchmark datasets (THUMOS'14 and TVSeries), outperforming all existing models used for comparison, with an mAP of 0.2% for THUMOS'14 and an mcAP of 0.1% for TVseries.
Online Handwritten Signature Verification Based on Temporal-Spatial Graph Attention Transformer
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China(多模态人工智能系统国家重点实验室(MAIS)、自动化研究所、中国科学院、北京100190,中国);School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China(人工智能学院、中国科学院大学、北京100049,中国)
LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attention
Beijing Institute of Technology(北京理工大学);Baidu Research(百度研究院);Inception Institute of Artificial Intelligence(人工智能起源研究所);National Engineering Laboratory of Deep Learning Technology and Application(深度学习技术及应用国家工程实验室);University of Kentucky(肯塔基大学)
TranStable: Towards Robust Pixel-level Online Video Stabilization by Jointing Transformer and CNN
Dalian University of Technology(大连理工大学);Institute for Clarity in Documentation(文档清晰度研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康分局);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
Image Recognition with Online Lightweight Vision Transformer: A Survey
Beijing University of Posts and Telecommunications(北京邮电大学);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology, Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science(计算能力网络与信息安全重点实验室,教育部,山东计算机科学中心,齐鲁大学,山东省计算能力互联网与服务计算重点实验室,山东省计算机科学基础研究中心)
3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking
Mercedes-Benz AG(梅赛德斯-奔驰股份公司);University of Bonn(波恩大学);Robert Bosch GmbH(罗伯特·博世有限公司);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所)
DORA: Dynamic Online Reinforcement Agent for Token Merging in Vision Transformers
University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥国家科学中心人工智能研究院)
ARGaze: Autoregressive Transformers for Online Egocentric Gaze Estimation
Department of Computer Science, University of Texas at Dallas, Richardson, TX, USA.(德克萨斯大学达拉斯分校计算机科学系);College of Computing, Georgia Institute of Technology, Atlanta, GA, USA(佐治亚理工学院计算机学院);Department of Computer Science, University of Wisconsin-Madison, Madison, WI, USA(威斯康星大学麦迪逊分校计算机科学系);Allen School of Computer Science, University of Washington, USA(华盛顿大学阿伦计算机科学学院)
Transformer-Driven Modeling of Variable Frequency Features for Classifying Student Engagement in Online Learning
Malaviya National Institute of Technology Jaipur(斋浦尔马维亚国家理工学院);Thapar Institute of Engineering and Technology(塔帕尔工程技术学院);SRM University AP(SRM 大学阿普拉分校)
Hyper-Decision Transformer for Efficient Online Policy Adaptation
Carnegie Mellon University(卡内基梅隆大学);University of Montreal(蒙特利尔大学);Mila(米拉人工智能研究所);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);UMass Amherst(马萨诸塞大学阿默斯特分校)
DORA: Dynamic Online Reinforcement Agent for Token Pruning in Vision Transformers
University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)
Visual-Word Tokenizer: Beyond Fixed Sets of Tokens in Vision Transformers
Predictive Analytics Lab, University of Sussex, UK(苏塞克斯大学预测分析实验室);University of Surrey, UK(塞夫尔大学);Basque Center for Applied Mathematics, Spain(巴斯克应用数学中心);Monash University, Indonesia(墨尔本大学)