In order to deal with variant-length long videos, prior works extract multi-modal features and fuse them to predict students' engagement intensity. In this paper, we present a new end-to-end method Class Attention in Video Transformer (CavT), which involves a single vector to process class embedding and to uniformly perform end-to-end learning on variant-length long videos and fixed-length short videos. Furthermore, to address the lack of sufficient samples, we propose a binary-order representatives sampling method (BorS) to add multiple video sequences of each video to augment the training set. BorS+CavT not only achieves the state-of-the-art MSE (0.0495) on the EmotiW-EP dataset, but also obtains the state-of-the-art MSE (0.0377) on the DAiSEE dataset. The code and models have been made publicly available at https://github.com/mountainai/cavt.
Attention-Guided Dual-Stream Learning for Group Engagement Recognition: Fusing Transformer-Encoded Motion Dynamics with Scene Context via Adaptive Gating
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Nanyang Technological University（南洋理工大学）；The University of Tokyo（东京大学）
This paper presents our approach for the engagement intensity regression task of EmotiW 2019. The task is to predict the engagement intensity value of a student when he or she is watching an online MOOCs video in various conditions. Based on our winner solution last year, we mainly explore head features and body features with a bootstrap strategy and two novel loss functions in this paper. We maintain the framework of multi-instance learning with long short-term memory (LSTM) network, and make three contributions. First, besides of the gaze and head pose features, we explore facial landmark fe…
Transformer-Driven Modeling of Variable Frequency Features for Classifying Student Engagement in Online Learning
Malaviya National Institute of Technology Jaipur（斋浦尔马维亚国家理工学院）；Thapar Institute of Engineering and Technology（塔帕尔工程技术学院）；SRM University AP（SRM 大学阿普拉分校）
NVIDIA Corporation（英伟达公司）；Seoul National University（首尔大学）
Synthesizing medical images, such as PET, is a challenging task due to the fact that the intensity range is much wider and denser than those in photographs and digital renderings and are often heavily biased toward zero. Above all, intensity values in PET have absolute significance, and are used to compute parameters that are reproducible across the population. Yet, usually much manual adjustment has to be made in pre-/post- processing when synthesizing PET images, because its intensity ranges can vary a lot, e.g., between -100 to 1000 in floating point values. To overcome these challenges, we…
From Remote Sensing to Multiple Time Horizons Forecasts: Transformers Model for CyanoHAB Intensity in Lake Champlain
Vermont Artificial Intelligence Laboratory (VaiL)（佛蒙特人工智能实验室（VaiL））；Department of Computer Science, University of Vermont（大学计算机科学系）；University of Vermont（佛蒙特大学）；Department of Community Development and Applied Economics & Department of Computer Science, University of Vermont（社区发展与应用经济学系及计算机科学系，佛蒙特大学）；Department of Geology, University of Vermont（地质学系，佛蒙特大学）；Department of Civil and Environmental Engineering, University of Vermont（土木与环境工程系，佛蒙特大学）；Vermont EPSCoR (Established Program to Stimulate Competitive Research), University of Vermont（佛蒙特EPSCoR（促进竞争性研究计划），佛蒙特大学）；Vermont Department of Environmental Conservation（佛蒙特州环境保护部）；Water Quality Solutions, Inc.（水质解决方案公司）
MambaLIE: Scene Light Intensity-Boosted Low-Light Image Enhancement with State Space Model
National and Local Joint Engineering Laboratory of Computer Aided Design, School of Software Engineering, Dalian University（大连大学软件工程学院计算机辅助设计国家地方联合工程实验室）；Hong Kong Polytechnic University（香港理工大学）；School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）；College of Life and Health, Dalian University（大连大学生命与健康科学学院）
Grading breast density is highly sensitive to normalization settings of digital mammogram as the density is tightly correlated with the distribution of pixel intensity. Also, the grade varies with readers due to uncertain grading criteria. These issues are inherent in the density assessment of digital mammography. They are problematic when designing a computer-aided prediction model for breast density and become worse if the data comes from multiple sites. In this paper, we proposed two novel deep learning techniques for breast density prediction: 1) photometric transformation which adaptively…
University of St Andrews（圣安德鲁斯大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Oulu（奥卢大学）
Being spontaneous, micro-expressions are useful in the inference of a person's true emotions even if an attempt is made to conceal them. Due to their short duration and low intensity, the recognition of micro-expressions is a difficult task in affective computing. The early work based on handcrafted spatio-temporal features which showed some promise, has recently been superseded by different deep learning approaches which now compete for the state of the art performance. Nevertheless, the problem of capturing both local and global spatio-temporal patterns remains challenging. To this end, here…
In fine-grained image recognition (FGIR), the localization and amplification of region attention is an important factor, which has been explored a lot by convolutional neural networks (CNNs) based approaches. The recently developed vision transformer (ViT) has achieved promising results on computer vision tasks. Compared with CNNs, Image sequentialization is a brand new manner. However, ViT is limited in its receptive field size and thus lacks local attention like CNNs due to the fixed size of its patches, and is unable to generate multi-scale features to learn discriminative region attention.…
Restoring Images in Adverse Weather Conditions via Histogram Transformer
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）；Guangdong Provincial Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Wechat Business Group, Tencent（腾讯微信事业群）
