The focus of this work is $\textit{sign spotting}$ - given a video of an isolated sign, our task is to identify $\textit{whether}$ and $\textit{where}$ it has been signed in a continuous, co-articulated sign language video. To achieve this sign spotting task, we train a model using multiple types of available supervision by: (1) $\textit{watching}$ existing footage which is sparsely labelled using mouthing cues; (2) $\textit{reading}$ associated subtitles (readily available translations of the signed content) which provide additional $\textit{weak-supervision}$; (3) $\textit{looking up}$ words (for which no co-articulated labelled examples are available) in visual sign language dictionaries to enable novel sign spotting. These three tasks are integrated into a unified learning framework using the principles of Noise Contrastive Estimation and Multiple Instance Learning. We validate the effectiveness of our approach on low-shot sign spotting benchmarks. In addition, we contribute a machine-readable British Sign Language (BSL) dictionary dataset of isolated signs, BSLDict, to facilitate study of this task. The dataset, models and code are available at our project page.
Towards Active Learning for Action Spotting in Association Football Videos
Curtin University(科廷大学);Football Technology & Innovation, FIFA(国际足联足球技术与创新部门);FIFA Medical, FIFA(国际足联医疗部门);Faculty of Medicine and Health, The University of Sydney(悉尼大学医学与健康学院)
OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos
Montefiore Institute, University of Liège(列日大学蒙泰菲奥里研究所);Cinfo & CITIC Research Center, University of A Coruña(拉科鲁尼亚大学Cinfo与CITIC研究中心);IVUL, KAUST(阿卜杜拉国王科技大学IVUL)
GoMatching++: Parameter- and Data-Efficient Arbitrary-Shaped Video Text Spotting and Benchmarking
School of Computer Science, National Engineering Research Center for Multimedia Software, and Institute of Artificial Intelligence, Wuhan University(计算机学院、多媒体软件国家工程研究中心及人工智能学院,武汉大学)
PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text
Nanjing University(南京大学);The University of Hong Kong(香港大学);Nanjing University of Science and Technology(南京理工大学);SenseTime Group Limited(商汤集团有限公司);Alibaba Group(阿里巴巴集团);Monash University(莫纳什大学)